public final class MLContextUtil extends Object
| Modifier and Type | Field and Description | 
|---|---|
| static Class[] | ALL_SUPPORTED_DATA_TYPESAll data types supported by the MLContext API. | 
| static Class[] | BASIC_DATA_TYPESBasic data types supported by the MLContext API. | 
| static String | BUILD_TIME_NOT_AVAILABLEBuild time not available message. | 
| static Class[] | COMPLEX_DATA_TYPESComplex data types supported by the MLContext API. | 
| static String | VERSION_NOT_AVAILABLEVersion not available message. | 
| Constructor and Description | 
|---|
| MLContextUtil() | 
| Modifier and Type | Method and Description | 
|---|---|
| static void | checkInputParameterType(String parameterName,
                       Object parameterValue)Verify that the type of input parameter value is supported. | 
| static void | checkInputValueType(String name,
                   Object value)Verify that the type of input value is supported. | 
| static void | checkInputValueTypes(Map<String,Object> inputs)Verify that the types of input values are supported. | 
| static Map<String,String> | convertInputParametersForParser(Map<String,Object> basicInputParameterMap)Converts non-string basic input parameter values to strings to pass to
 the parser. | 
| static Data | convertInputType(String parameterName,
                Object parameterValue)Convert input types to internal SystemDS representations | 
| static Data | convertInputType(String parameterName,
                Object parameterValue,
                Metadata metadata)Convert input types to internal SystemDS representations | 
| static void | deleteRemoveVariableInstructions(Program progam)Delete the 'remove variable' instructions from a runtime program. | 
| static String | determineOutputTypeAsString(LocalVariableMap symbolTable,
                           String outputName)Obtain a symbol table output type as a String | 
| static String | displayInputs(String name,
             Map<String,Object> map,
             LocalVariableMap symbolTable)Obtain a display of script inputs. | 
| static String | displayMap(String mapName,
          Map<String,Object> map)Display the keys and values in a Map | 
| static String | displayOutputs(Set<String> outputNames,
              LocalVariableMap symbolTable)Obtain a display of the script outputs. | 
| static String | displayOutputs(String name,
              Set<String> outputNames,
              LocalVariableMap symbolTable)Obtain a display of the script outputs. | 
| static String | displaySet(String setName,
          Set<String> set)Display the values in a Set | 
| static String | displaySymbolTable(LocalVariableMap symbolTable)Display the keys and values in the symbol table | 
| static String | displaySymbolTable(String name,
                  LocalVariableMap symbolTable)Display the keys and values in the symbol table | 
| static boolean | doesDataFrameLookLikeMatrix(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)Examine the DataFrame schema to determine whether the data appears to be
 a matrix. | 
| static boolean | doesSymbolTableContainFrameObject(LocalVariableMap symbolTable,
                                 String variableName)Determine if the symbol table contains a FrameObject with the given
 variable name. | 
| static boolean | doesSymbolTableContainMatrixObject(LocalVariableMap symbolTable,
                                  String variableName)Determine if the symbol table contains a MatrixObject with the given
 variable name. | 
| static String | getBasicTypeString(Object object)Obtain the SystemDS scalar value type string equivalent of an accepted
 basic type (Integer, Boolean, Double, String) | 
| static org.apache.spark.api.java.JavaSparkContext | getJavaSparkContext(MLContext mlContext)Obtain the Java Spark Context | 
| static org.apache.spark.api.java.JavaSparkContext | getJavaSparkContextFromProxy()Obtain the Java Spark Context from the MLContextProxy | 
| static org.apache.spark.SparkContext | getSparkContext(MLContext mlContext)Obtain the Spark Context | 
| static org.apache.spark.SparkContext | getSparkContextFromProxy()Obtain the Spark Context from the MLContextProxy | 
| static org.apache.spark.sql.SparkSession | getSparkSessionFromProxy()Obtain the Spark Session from the MLContextProxy | 
| static boolean | isBasicType(Object object)Is the object one of the supported basic data types? (Integer, Boolean,
 Double, String) | 
| static boolean | isComplexType(Object object)Is the object one of the supported complex data types? (JavaRDD, RDD,
 DataFrame, Matrix, double[][], MatrixBlock, URL) | 
| static boolean | isCSVLineAllNumbers(String line)If no metadata is supplied for an RDD or JavaRDD, this method can be used
 to determine whether the data appears to be matrix (or a frame) | 
| static boolean | isSparkVersionSupported(String sparkVersion,
                       String minimumRecommendedSparkVersion)Determine whether the Spark version is supported. | 
| static String | quotedString(String str)Return a double-quoted string with inner single and double quotes
 escaped. | 
| static void | setCompilerConfig()Set SystemDS compiler configuration properties for MLContext | 
| static void | setConfig(String configFilePath)Set SystemDS configuration properties based on a configuration file. | 
| static void | setDefaultConfig()Set default SystemDS configuration properties. | 
| static void | verifySparkVersionSupported(org.apache.spark.sql.SparkSession spark)Check that the Spark version is supported. | 
| static String | welcomeMessage()The SystemDS welcome message | 
public static final String VERSION_NOT_AVAILABLE
public static final String BUILD_TIME_NOT_AVAILABLE
public static final Class[] BASIC_DATA_TYPES
public static final Class[] COMPLEX_DATA_TYPES
public static final Class[] ALL_SUPPORTED_DATA_TYPES
public static boolean isSparkVersionSupported(String sparkVersion, String minimumRecommendedSparkVersion)
sparkVersion - Spark version string (ie, "1.5.0").minimumRecommendedSparkVersion - Minimum recommended Spark version string (ie, "2.1.0").true if Spark version supported; otherwise false.public static void verifySparkVersionSupported(org.apache.spark.sql.SparkSession spark)
spark - SparkSessionMLContextException - thrown if Spark version isn't supportedpublic static void setDefaultConfig()
public static void setConfig(String configFilePath)
configFilePath - Path to configuration file.MLContextException - if configuration file was not found or a parse exception
             occurredpublic static void setCompilerConfig()
public static void checkInputValueTypes(Map<String,Object> inputs)
inputs - Map of String/Object pairsMLContextException - if an input value type is not supportedpublic static void checkInputValueType(String name, Object value)
name - The name of the inputvalue - The value of the inputMLContextException - if the input value type is not supportedpublic static void checkInputParameterType(String parameterName, Object parameterValue)
parameterName - The name of the input parameterparameterValue - The value of the input parameterMLContextException - if the input parameter value type is not supportedpublic static boolean isBasicType(Object object)
object - the object type to be examinedtrue if type is a basic data type; otherwise
         false.public static String getBasicTypeString(Object object)
object - the object type to be examinedpublic static boolean isComplexType(Object object)
object - the object type to be examinedtrue if type is a complex data type; otherwise
         false.public static Map<String,String> convertInputParametersForParser(Map<String,Object> basicInputParameterMap)
basicInputParameterMap - map of input parameterspublic static Data convertInputType(String parameterName, Object parameterValue)
parameterName - The name of the input parameterparameterValue - The value of the input parameterpublic static Data convertInputType(String parameterName, Object parameterValue, Metadata metadata)
parameterName - The name of the input parameterparameterValue - The value of the input parametermetadata - matrix/frame metadatapublic static boolean isCSVLineAllNumbers(String line)
line - a line of the RDDtrue if all the csv-separated values are numbers,
         false otherwisepublic static boolean doesDataFrameLookLikeMatrix(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)
df - the DataFrametrue if the DataFrame appears to be a matrix,
         false otherwisepublic static String quotedString(String str)
str - the original stringpublic static String displayMap(String mapName, Map<String,Object> map)
mapName - the name of the mapmap - Map of String keys and Object valuespublic static String displaySet(String setName, Set<String> set)
setName - the name of the Setset - Set of String valuespublic static String displaySymbolTable(String name, LocalVariableMap symbolTable)
name - the name of the symbol tablesymbolTable - the LocalVariableMappublic static String displaySymbolTable(LocalVariableMap symbolTable)
symbolTable - the LocalVariableMappublic static String determineOutputTypeAsString(LocalVariableMap symbolTable, String outputName)
symbolTable - the symbol tableoutputName - the name of the output variablepublic static String displayInputs(String name, Map<String,Object> map, LocalVariableMap symbolTable)
name - the title to display for the inputsmap - the map of inputssymbolTable - the symbol tablepublic static String displayOutputs(String name, Set<String> outputNames, LocalVariableMap symbolTable)
name - the title to display for the outputsoutputNames - the names of the output variablessymbolTable - the symbol tablepublic static String displayOutputs(Set<String> outputNames, LocalVariableMap symbolTable)
outputNames - the names of the output variablessymbolTable - the symbol tablepublic static String welcomeMessage()
public static org.apache.spark.SparkContext getSparkContext(MLContext mlContext)
mlContext - the SystemDS MLContextpublic static org.apache.spark.api.java.JavaSparkContext getJavaSparkContext(MLContext mlContext)
mlContext - the SystemDS MLContextpublic static org.apache.spark.SparkContext getSparkContextFromProxy()
public static org.apache.spark.api.java.JavaSparkContext getJavaSparkContextFromProxy()
public static org.apache.spark.sql.SparkSession getSparkSessionFromProxy()
public static boolean doesSymbolTableContainFrameObject(LocalVariableMap symbolTable, String variableName)
symbolTable - the LocalVariableMapvariableName - the variable nametrue if the variable in the symbol table is a
         FrameObject, false otherwise.public static boolean doesSymbolTableContainMatrixObject(LocalVariableMap symbolTable, String variableName)
symbolTable - the LocalVariableMapvariableName - the variable nametrue if the variable in the symbol table is a
         MatrixObject, false otherwise.public static void deleteRemoveVariableInstructions(Program progam)
progam - runtime programCopyright © 2020 The Apache Software Foundation. All rights reserved.