home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 123971294

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
123971294 MDU6SXNzdWUxMjM5NzEyOTQ= 684 to_masked_array should have an option to always return array 5049737 closed 0     1 2015-12-27T10:15:59Z 2015-12-27T19:40:02Z 2015-12-27T19:40:02Z MEMBER      

Hello,

I'm working with Monary and xray and faced a problem because xray.DataArray.to_masked_array doesn't return an array of boolean but just a boolean.

``` In [1]: ma.masked_array(df['Bid'].values, df['Bid'].isnull()) Out[1]: masked_array(data = [0.81 0.88805 0.8880899999999999 ..., 0.87531 0.87531 0.87531], mask = [False False False ..., False False False], fill_value = 1e+20)

In [2]: ds.Bid.to_masked_array() Out[2]: masked_array(data = [ 0.81 0.88805 0.88809 ..., 0.87531 0.87531 0.87531], mask = False, fill_value = 1e+20) ```

So we have

mask = False,

instead of

mask = [False False False ..., False False False],

It will be nice if a parameter could be passed to always return array.

Here is my code With sample data from https://drive.google.com/file/d/0B8iUtWjZOTqla3ZZTC1FS0pkZXc/view?usp=sharing

import pandas as pd import xray df = pd.read_csv("AUDUSD-2014-01.csv", names=['Symbol', 'Date', 'Bid', 'Ask']) df['Date'] = pd.to_datetime(df['Date'], format='%Y%m%d %H:%M:%S.%f') ds = xray.Dataset.from_dataframe(df) ds = ds[['Bid', 'Ask']] lst_cols = list(map(lambda col: ds[col].to_masked_array(), ds.data_vars)) mparams = monary.MonaryParam.from_lists(lst_cols, list(ds.data_vars), ['float64', 'float64']) m.insert('monary_db', 'ticks', mparams)

It raises

``` //anaconda/lib/python3.4/site-packages/monary/monary.py in insert(self, db, coll, params, write_concern) 763 for i, param in enumerate(params): 764 data_p = param.array.data.ctypes.data_as(ctypes.c_void_p) --> 765 mask_p = param.array.mask.ctypes.data_as(ctypes.c_void_p) 766 767 if cmonary.monary_set_column_item(

AttributeError: 'numpy.bool_' object has no attribute 'ctypes' ```

see lst_cols

[masked_array(data = [ 0.88796 0.88805 0.88809 ..., 0.87531 0.87531 0.87531], mask = False, fill_value = 1e+20), masked_array(data = [ 0.88922 0.88914 0.8891 ..., 0.87588 0.87574 0.87574], mask = False, fill_value = 1e+20)]

An other option could be to fix this on Monary side https://bitbucket.org/djcbeach/monary/issues/22/insert-doesnt-work-correctly-when-mask-is

Kind regards

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/684/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 0 rows from issues_id in issues_labels
  • 1 row from issue in issue_comments
Powered by Datasette · Queries took 0.749ms · About: xarray-datasette