You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After attempting to copy the sample code from the readme, I get a compilation error when trying to call the RegisterTempTable method on a data frame.
var sparkConf = new SparkConf();
sparkConf.SetMaster("local[2]"); //local standalone with 2 cores, ex:"spark://master:7077" to connect to other cluster
sparkConf.SetAppName("SparkCLRExample");
//sparkConf.SetSparkHome(""); //used to set the spark install location on worker nodes. Using default instead.
SparkContext sc = SparkContext.GetOrCreate(sparkConf);
var reqDataFrame = sc.TextFile(@"hdfs://path/to/requests.csv");
var metricDataFrame = sc.TextFile(@"hdfs://path/to/metrics.csv");
reqDataFrame.RegisterTempTable("requests");
metricDataFrame.RegisterTempTable("metrics");
// C0 - guid in requests DataFrame, C3 - guid in metrics DataFrame
var joinDataFrame = GetSqlContext().Sql(
"SELECT joinedtable.datacenter" +
", MAX(joinedtable.latency) maxlatency" +
", AVG(joinedtable.latency) avglatency " +
"FROM (" +
"SELECT a.C1 as datacenter, b.C6 as latency " +
"FROM requests a JOIN metrics b ON a.C0 = b.C3) joinedtable " +
"GROUP BY datacenter");
joinDataFrame.ShowSchema();
joinDataFrame.Show();
'RDD' does not contain a definition for 'RegisterTempTable' and no extension method 'RegisterTempTable' accepting a first argument of type 'RDD' could be found (press F4 to add a using directive or assembly reference)
The text was updated successfully, but these errors were encountered:
After attempting to copy the sample code from the readme, I get a compilation error when trying to call the RegisterTempTable method on a data frame.
The text was updated successfully, but these errors were encountered: