issues: 262696381
This data as json
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
262696381 | MDU6SXNzdWUyNjI2OTYzODE= | 1604 | Where functionality in xarray including else case (dask compability) | 30219501 | closed | 0 | 8 | 2017-10-04T07:51:39Z | 2018-06-13T14:57:25Z | 2017-12-14T17:49:56Z | NONE | I am faced with the flexibility needed to compute different types of skill scores using xarray. Thus, keeping in mind the attached code - a method for computing a modified mean squared error skill score ("AVSS") - I am fighting with the following problems: 1. I want to try to keep the code user-friendly regarding an extension of my program to other skill scores. Thus, the middle part of the attached method utilizing the if-then-else statement shall be outsourced. 2. There are three input datasets in case of skill scores: self.DSref = observations, self.DSrefmod = reference model, self.proof = model to evaluate. I have to combine all three with simple arithmetics (minus), but xarray does not allow simple arithmetics in case of small differences in the coordinates between the three datasets (also if the data type of the coordinates differ from float64 to float). Thus, my horrifying workaround is to make a loop over all variables I want to evaluate and to do for each variable the following: a) create a new dataset "DSnew" based on the dataset-variable "self.DSproof[varnsproof]", b) rename the variable in "DSnew" to the variable name I want to have for the evaulation result (e.g. Bias of temperature or skill score of temperature), c) create some help variables "DSnew['MSE_p1]" by copying and d) modifying the data of the variables to compute those mathematical operations of the related skill score invariant to temporal aggregation, e) applying grouping and resampling to compute climate statistics as monthly means or daily cycles and f) final mathematical operation of the related skill score which has to be done after temporal aggregation. Is there a better way to handle the operations / to prevent the strange process of creating new datasets and copying variables and to prevent the outer loop over the variables? What would be your short code to handle my problem? 3. The where functionality is sometimes needed to compute skill scores. I have used the where function of numpy, but as I read in your xarray-documentation, an explicit call of numpy functions is not compatible with dask-arrays? Is there an analogue in the xarray-package? ``` def squarefunc(x): return xarray.ufuncs.square(x) def AVSS_def(x): AVSS_p1 = x["MSE_p1"]/x["MSE_p2"] * (-1.0) + 1.0 AVSS_p2 = x["MSE_p2"]/x["MSE_p1"] - 1.0 x[varnsres].data = np.where( (x["MSE_p2"] - x["MSE_p1"]) > 0,AVSS_p1,AVSS_p2 ) return x endresult = xarray.Dataset() for varnsrefmod,varnsproof,varnsref,varnsres in zip(self.varns_refmod,self.varns_proof,self.varns_ref,varns_result): DSnew = xarray.merge([xarray.Dataset(),self.DSproof[varnsproof]]) DSnew.rename({varnsproof : varnsres },inplace=True) DSnew["MSE_p1"] = DSnew[varnsres].copy() DSnew["MSE_p2"] = DSnew[varnsres].copy() DSnew["MSE_p1"].data = squarefunc(self.DSproof[varnsproof].data - self.DSref[varnsref].data) DSnew["MSE_p2"].data = squarefunc(self.DSrefmod[varnsrefmod].data - self.DSref[varnsref].data) coordtime = GeneralUtils.FromDimList2Pyxarray(dim_time[varnsref]) if aggregtime == 'fullperiod': DSnew = DSnew.mean(coordtime); self.RepairTime.update({'Needed' : False}); elif aggregtime == '-': DSnew = DSnew; self.RepairTime.update({'Needed' : False}); elif "overyears" in aggregtime: grpby_method=GeneralUtils.ConvertAggregationKey2XRgroupby(aggregtime) DSnew = DSnew.groupby(coordtime+'.'+grpby_method).mean(coordtime); self.RepairTime.update({'Needed' : True}); self.RepairTime.update({'start' : self.DSref[coordtime].data[0] }); self.RepairTime.update({'end' : self.DSref[coordtime].data[-1]}) elif "overyears" not in aggregtime: resamplefreq=GeneralUtils.ConvertAggregationKey2Resample(aggregtime) DSnew = DSnew.resample(resamplefreq, dim=coordtime, how='mean'); self.RepairTime.update({'Needed' : False}); AVSS_def(DSnew); self.Update_Attributes(Datasetobj=DSnew,variable=varnsres,stdname=varnsres,units=self.DSref[varnsref].attrs['units'], \ longname="temporal AVSS of "+self.DSref[varnsref].attrs['long_name']) endresult = xarray.merge([endresult,DSnew]) ``` |
{ "url": "https://api.github.com/repos/pydata/xarray/issues/1604/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | 13221727 | issue |