issue_comments

Strange... thanks for checking again.

OK, on the current master and with a cartopy installed from conda-forge I am not able to reproduce the problem any more:

imshow xray: 3.30 s pcolormesh xray: 13.13 s pcolormesh cartopy: 12.85 s

I don't really understand what happened in between... The factor 4 between imshow and pcolormesh remains, but at least the show-killer 30sec plotting time I had before is gone. I'm closing this now, and this will remain a mystery.

(note: on my laptop I'm getting faster plotting results in a virtualenv configured with pip install than with pandas...)

I had little time to spend on this lately but I'll try to get back to it in the next days. Any idea on where it could come from? I found it quite hard to debug because of the many decorators... From the profiling and the number of function calls above there seems to be something quite big happening in between xray and cartopy. Could it be something as trivial as a double function call or something?

See the number of calls:

faster.txt: 4099953 function calls (4093947 primitive calls) in 14.571 seconds

slower.txt: 10316809 function calls (10303964 primitive calls) in 34.330 seconds

Profiling for each version of pcolormesh: https://gist.github.com/shoyer/73e3841827fe1eb08d00

Switching a masked array doesn't seem to make the non-xray version any slower...

Changing the input to np.ma.asarray(l1 + l2) did not change anything as far as I can see...

I can't verify right now but it may have something to do with using masked arrays under the hood. There are no nan's in your example but xray still is converting the array to a masked_array before plotting. I bet plotting with pcolormesh is slower with masked arrays than with numpy arrays.

Yes, I'm seeing the same thing. Very weird -- I'll see if I can profile it.

On Wed, Nov 18, 2015 at 8:28 AM, Fabien Maussion notifications@github.com wrote:

Was someone able to reproduce this or is it just me?

Reply to this email directly or view it on GitHub: https://github.com/xray/xray/issues/657#issuecomment-157768356

Was someone able to reproduce this or is it just me?

This is surprising! But good to know.

Thanks @pelson ! So now this is when it becomes funny: I've been able to make three similar plots using xray's imshow, pcolormesh and cartopy's pcolormesh. Xray's pcolormesh takes twice the time as cartopy's:

``` python import matplotlib.pyplot as plt from matplotlib.colors import Normalize import xray import numpy as np import cartopy.crs as ccrs import time

nlats, nlons = (241, 480) lats = np.linspace(90, -90, nlats) lons = np.linspace(0, 360-0.75, nlons) l1, l2 = np.meshgrid(lons, lats) data = xray.DataArray(l1 + l2, [('latitude', lats), ('longitude', lons)])

cmap = plt.get_cmap('viridis') norm = Normalize(vmin=0, vmax=data.max().values)

start_time = time.time() fig = plt.figure() ax = plt.axes(projection=ccrs.Robinson()) data.plot.imshow(ax=ax, transform=ccrs.PlateCarree(), add_colorbar=False, cmap=cmap, vmin=0) ax.coastlines() plt.savefig('imshow_xray.png') print("imshow xray: {:.2f} s".format(time.time() - start_time))

start_time = time.time() fig = plt.figure() ax = plt.axes(projection=ccrs.Robinson()) data.plot.pcolormesh(ax=ax, transform=ccrs.PlateCarree(), add_colorbar=False, cmap=cmap, vmin=0) ax.coastlines() plt.savefig('pcolormesh_xray.png') print("pcolormesh xray: {:.2f} s".format(time.time() - start_time))

start_time = time.time() fig = plt.figure() ax = plt.axes(projection=ccrs.Robinson()) ax.pcolormesh(lons, lats, l1 + l2, transform=ccrs.PlateCarree(), cmap=cmap, norm=norm) ax.coastlines() plt.savefig('pcolormesh_cartopy.png') print("pcolormesh cartopy: {:.2f} s".format(time.time() - start_time))

```

imshow xray: 3.06 s pcolormesh xray: 27.50 s pcolormesh cartopy: 12.14 s

Thanks @fmaussion - I've raised it in https://github.com/SciTools/cartopy/issues/700. Thanks for putting together the self contained example - it's fine to have xray as a dependency on that. FWIW you would do something like plt.pcolormesh(lons, lats, l1 + l2, transform=ccrs.PlateCarree()) if you didn't have a DataArray.

Hi again,

I've made a self-contained example below. I have no clue about how cartopy works (blind xray user, sorry :-() and was not able to remove the dependency to xray without getting different plots for imshow() and pcolormesh()... If one of the xray gurus could help me to remove the xray part of the code I could open an issue in cartopy. I wonder if the problem comes from the fact that ERA-Interim lons are spanning 0-360?

``` python import matplotlib.pyplot as plt import xray import numpy as np import cartopy.crs as ccrs import time

nlats, nlons = (241, 480) lats = np.linspace(90, -90, nlats) lons = np.linspace(0, 360-0.75, nlons) l1, l2 = np.meshgrid(lons, lats) data = xray.DataArray(l1 + l2, [('latitude', lats), ('longitude', lons)])

start_time = time.time() fig = plt.figure() ax = plt.axes(projection=ccrs.Robinson()) data.plot.imshow(ax=ax, transform=ccrs.PlateCarree()) ax.coastlines() plt.savefig('imshow.png') print("imshow: {:.2f} s".format(time.time() - start_time))

start_time = time.time() fig = plt.figure() ax = plt.axes(projection=ccrs.Robinson()) data.plot.pcolormesh(ax=ax, transform=ccrs.PlateCarree()) ax.coastlines() plt.savefig('pcolormesh.png') print("pcolormesh: {:.2f} s".format(time.time() - start_time)) ```

imshow: 3.09 s pcolormesh: 27.53 s

There is definitely scope for being smarter with cartopy's pcolormesh. There isn't an issue for it yet, but would be happy if you opened up a performance related issue in cartopy. pcolormesh will always be slower than imshow, but in most cases, not an order of magnitude slower!

I guess it's OK to leave it as is, as long as the performance loss is documented. I like the automatic plot() function because it hides the boring mpl internals to my students and (until now) always did what I expected it to do.

30 seconds to generate a plot in a Notebook is definitely a fun killer. Maybe I should still upgrade to 0.6.1 and explain to my students that they should use imshow() instead of plot().

Yes, the difference in speed is definitely due to cartopy, which handles the projections. @pelson may be able to clarify whether imshow is always slower than pcolormesh when using cartopy, or if that is specific to particular projections.

We changed the default from imshow to pcolormesh in #608. I would not be opposed to changing the default back, given that we already product essentially equivalent plots with both methods. We did add a performance note to our (currently broken) docs: http://xray.readthedocs.org/en/latest/plotting.html#two-dimensions

@jhamman and/or @clarkfitzg may have some opinions here.

If I remove the projection stuff imshow and pcolormesh are equally fast. Both functions must handle the projection differently, somehow?

Thanks for the quick answer. changing to imshow indeed reduced the execution time back to normal. I wouldn't expect a factor 10 between imshow and pcolormesh, though...

We changed the default plot type from "imshow" in xray v0.6 to "pcolormesh" in xray v0.6.1. Does setting that manually make a difference? e.g., t2_avg.plot.imshow?