在jupyter筆記本（pyspark）中使用Seaborn時出錯

Question

我試圖使用Seaborn將數據可視化。我已經在pyspark中使用SQLContext創建了一個數據框。但是，當我調用lmplot時會導致錯誤。我不知道我錯過了什麼。下面給出的是我的代碼（我使用jupyter筆記本）：

import pandas as pd 

from matplotlib import pyplot as plt 

import seaborn as sns 

from pyspark.sql import SQLContext 

sqlContext = SQLContext(sc) 

df = sqlContext.read.load('file:///home/cloudera/Downloads/WA_Sales_Products_2012-14.csv', 
         format='com.databricks.spark.csv', 
         header='true',inferSchema='true') 

sns.lmplot(x='Quantity', y='Year', data=df) 

Error trace: 
--------------------------------------------------------------------------- 
TypeError         Traceback (most recent call last) 
<ipython-input-86-2a2b43993475> in <module>() 
----> 2 sns.lmplot(x='Quantity', y='Year', data=df) 

/home/cloudera/anaconda3/lib/python3.5/site-packages/seaborn/regression.py in lmplot(x, y, data, hue, col, row, palette, col_wrap, size, aspect, markers, sharex, sharey, hue_order, col_order, row_order, legend, legend_out, x_estimator, x_bins, x_ci, scatter, fit_reg, ci, n_boot, units, order, logistic, lowess, robust, logx, x_partial, y_partial, truncate, x_jitter, y_jitter, scatter_kws, line_kws) 
    557      hue_order=hue_order, size=size, aspect=aspect, 
    558      col_wrap=col_wrap, sharex=sharex, sharey=sharey, 
--> 559      legend_out=legend_out) 
    560 
    561  # Add the markers here as FacetGrid has figured out how many levels of the 

/home/cloudera/anaconda3/lib/python3.5/site-packages/seaborn/axisgrid.py in __init__(self, data, row, col, hue, col_wrap, sharex, sharey, size, aspect, palette, row_order, col_order, hue_order, hue_kws, dropna, legend_out, despine, margin_titles, xlim, ylim, subplot_kws, gridspec_kws) 
    255   # Make a boolean mask that is True anywhere there is an NA 
    256   # value in one of the faceting variables, but only if dropna is True 
--> 257   none_na = np.zeros(len(data), np.bool) 
    258   if dropna: 
    259    row_na = none_na if row is None else data[row].isnull() 

TypeError: object of type 'DataFrame' has no len() 

任何幫助或指針表示讚賞。預先感謝您:-)

Answer 1

sqlContext.read.load(...)返回一個Spark-DataFrame。我不確定，seaborn是否可以自動將Spark-DataFrame轉換爲Pandas-Dataframe。

嘗試：

sns.lmplot(x='Quantity', y='Year', data=df.toPandas()) 

df.toPandas()從星火-DF返回大熊貓-DF。