issue_comments

@shoyer i've tested the solution provided, it works like a charm with my tests however many tests are broken on test_backends.py with cases where we lose precision i'll give you more detail.

@shoyer sorry for the delayed response. dtypes.result_type(1, np.float32(1)) returns dtype('float64') That's what makes this behaviour for Python's int and float Keeping the consistency would then require testing if scale_factor*var_dtype + add_offset fit in var_dtype in case of Python's int and float. Correct me if i'm wrong but this is a bit hard to do without knowing max,min values to avoid overlapping Evaluating those could break performance if only used for encoding and decoding. Do have any idea how this could be acheived ? or is it simpler to keep it as it is ?

@shoyer do you mean that we consider that by default when we deal with Python's int and float we cast them to np.int64 and np.float64 ?

@shoyer tests are failing but it does'nt seem to be coming from this PR, i saw the same error on other PRs as well, my tests were working fine until i made a git pull.

@shoyer did you have a look at this ?

@shoyer i changed the implementation, and took into consideration your comments. now returning largest type takes place only when decoding. i added a test with all types of scale factor, add_offset and variable.

@shoyer now scale_factor and add_offset are taken into account when encoding and decoding data. if none of them is present or if they're not a subtype of np.generic the old behaviour takes place.

@shoyer yes sure i'll update the pull request with the mentioned modifications.

@shoyer the logic is now propagated down to _choose_float_dtype inside CFScaleOffsetCoder, please let me know what you think.

@shoyer did you have a look at this ?

Hi, thank you for your effort into making xarray a great library. As mentioned in the issue the discussion went on a PR in order to make xr.open_dataset parametrable. This post is about asking you about recommendations regarding our PR.

In this case we would add a parameter to the open_dataset function called "force_promote" which is a boolean and False by default and thus not mandatory. And then spread that parameter down to the function maybe_promote in dtypes.py Where we say the following:

if dtype.itemsize <= 2 and not force_promote: dtype = np.float32 else: dtype = np.float64

The downside of that is that we somehow pollute the code with a parameter that is used in a specific case.

The second approach would check the value of an environment variable called "XARRAY_FORCE_PROMOTE" if it exists and set to true would force promoting type to float64.

please tells us which approach suits best your vision of xarray.

Regards.

Any updates about this ?

As mentioned in the original issue the modification is straightforward. Any ideas if this could be integrated to xarray anytime soon ?

Thank you for your quick answer. In our case we could evaluate std dev or square sums on long lists of values and the accumulation of those small values due to float32 type could create considerable differences.