Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

errors saving dataframes since 1.8.0 #118

Closed
mihailescu2m opened this issue Jun 27, 2020 · 5 comments · Fixed by #123
Closed

errors saving dataframes since 1.8.0 #118

mihailescu2m opened this issue Jun 27, 2020 · 5 comments · Fixed by #123
Assignees
@rolincova
Labels
bug Something isn't working
Milestone
1.9.0

Comments

@mihailescu2m
Copy link

mihailescu2m commented Jun 27, 2020 •
edited
Loading

I am saving a pandas dataframe like this:

db = InfluxDBClient(url=config.DB_URL, token=config.DB_TOKEN, org=config.DB_ORG
db_write_api = db.write_api()
db_write_api.write(config.DB_BUCKET, record=df, data_frame_measurement_name=price_history)
db_write_api.__del__()
db.__del__()

With 1.7.0 it worked well.
With 1.8.0 I am getting errors about the date:

The batch item wasn't processed successfully because: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json; charset=utf-8', 'X-Platform-Error-Code': 'invalid', 'Date': 'Sat, 27 Jun 2020 21:50:02 GMT', 'Transfer-Encoding': 'chunked'})
HTTP response body: {"code":"invalid","message":"unable to parse 'price_history daily_charge=106.660794520548,actual_kw_price=4.4495000000000005,forecast_kw_price=nan,actual_general_use=21.906245000000002,forecast_general_use=nan,actual_controlled_load=15.229245000000002,forecast_controlled_load=nan,actual_fit=-6.999245000000001,forecast_fit=nan,actual_renewables=25.036933442426097,forecast_renewables=nan 1593207000000000000': invalid number\nunable to parse 'price_history ..... [error continues for each row]
@bednar
Copy link
Contributor

bednar commented Jun 28, 2020

@mihailescu2m thanks, we will take a look

@bednar bednar added bug Something isn't working state: in progress labels Jun 29, 2020
@bednar
Copy link
Contributor

bednar commented Jun 29, 2020

@mihailescu2m could you please share how your Pandas DataFrame looks like?

@mihailescu2m
Copy link
Author

mihailescu2m commented Jun 29, 2020 •
edited
Loading

@bednar is this enough?

print(df)
                     daily_charge  actual_kw_price  forecast_kw_price  actual_general_use  forecast_general_use  actual_controlled_load  forecast_controlled_load  actual_fit  forecast_fit  actual_renewables  forecast_renewables
ts                                                                                                                                                                                                                                 
2020-06-28 18:30:00    106.660795           1.7347                NaN           18.892817                   NaN               12.215817                       NaN   -3.985817           NaN          95.838662                  NaN
2020-06-28 19:00:00    106.660795           2.6631                NaN           19.923341                   NaN               13.246341                       NaN   -5.016341           NaN          90.276395                  NaN
2020-06-28 19:30:00    106.660795           3.1955                NaN           20.514305                   NaN               13.837305                       NaN   -5.607305           NaN          83.388326                  NaN
2020-06-28 20:00:00    106.660795           4.5749                NaN           22.045439                   NaN               15.368439                       NaN   -7.138439           NaN          68.391742                  NaN
2020-06-28 20:30:00    106.660795           5.7310                NaN           23.328710                   NaN               16.651710                       NaN   -8.421710           NaN          59.522250                  NaN
...                           ...              ...                ...                 ...                   ...                     ...                       ...         ...           ...                ...                  ...
2020-06-30 15:30:00    106.660795              NaN           3.582510                 NaN             20.943886                     NaN                 14.266886         NaN     -6.036886                NaN            57.748487
2020-06-30 16:00:00    106.660795              NaN           3.412366                 NaN             20.755026                     NaN                 14.078026         NaN     -5.848026                NaN            63.197402
2020-06-30 16:30:00    106.660795              NaN           3.138664                 NaN             20.451217                     NaN                 13.774217         NaN     -5.544217                NaN            70.247935
2020-06-30 17:00:00    106.660795              NaN           2.544090                 NaN             19.791240                     NaN                 13.114240         NaN     -4.884240                NaN            75.524903
2020-06-30 17:30:00    106.660795              NaN           2.303061                 NaN             19.523698                     NaN                 12.846698         NaN     -4.616698                NaN            81.681978

[95 rows x 11 columns]

print(df.info())
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 95 entries, 2020-06-28 18:30:00 to 2020-06-30 17:30:00
Data columns (total 11 columns):
 #   Column                    Non-Null Count  Dtype  
---  ------                    --------------  -----  
 0   daily_charge              95 non-null     float64
 1   actual_kw_price           49 non-null     float64
 2   forecast_kw_price         46 non-null     float64
 3   actual_general_use        49 non-null     float64
 4   forecast_general_use      46 non-null     float64
 5   actual_controlled_load    49 non-null     float64
 6   forecast_controlled_load  46 non-null     float64
 7   actual_fit                49 non-null     float64
 8   forecast_fit              46 non-null     float64
 9   actual_renewables         49 non-null     float64
 10  forecast_renewables       46 non-null     float64
dtypes: float64(11)
memory usage: 8.9 KB
None

@bednar
Copy link
Contributor

bednar commented Jun 30, 2020

It is perfect. Thanks

rolincova added a commit that referenced this issue Jul 1, 2020
@rolincova
fix: Fixed serialization of DataFrame with empty values #118
bb8e665
rolincova added a commit that referenced this issue Jul 1, 2020
@rolincova
fix: Fixed serialization of DataFrame with empty values #118
e8300f7
rolincova added a commit that referenced this issue Jul 1, 2020
@rolincova
fix: Fixed serialization of DataFrame with empty values #118
6455ada
rolincova added a commit that referenced this issue Jul 2, 2020
@rolincova
fix: Fixed serialization of DataFrame with empty (NaN) values #118
839a542
rolincova added a commit that referenced this issue Jul 2, 2020
@rolincova
fix: Fixed serialization of DataFrame with empty (NaN) values #118
eedf2cd
rolincova added a commit that referenced this issue Jul 3, 2020
@rolincova
fix: Fixed serialization of DataFrame with empty (NaN) values #118
108ad18
@rolincova rolincova mentioned this issue Jul 3, 2020
Merged
7 tasks
rolincova added a commit that referenced this issue Jul 3, 2020
@rolincova
fix: Fixed serialization of DataFrame with empty (NaN) values #118
1255a0a
rolincova added a commit that referenced this issue Jul 13, 2020
@rolincova
fix: Fixed serialization of DataFrame with empty (NaN) values #118
a014150
rolincova added a commit that referenced this issue Jul 13, 2020
@rolincova
fix: Fixed serialization of DataFrame with empty (NaN) values #118
cd01183
@bednar bednar added this to the 1.9.0 milestone Jul 13, 2020
@bednar
Copy link
Contributor

bednar commented Jul 13, 2020

Hi @mihailescu2m ,

The issue is fixed in 1.9.0 milestone.

If you would like to use a dev version then install client via:

pip install git+https://github.com/influxdata/influxdb-client-python.git@master

Regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rolincova rolincova

Labels
bug Something isn't working
Projects
None yet
Milestone
1.9.0
3 participants
@bednar @mihailescu2m @rolincova