How not to organize your radar data files

So, in this PhD thingy I have some radar data files to work with. Not too hard, is it? A single radar beam, which records data in N-second batches. So we get a nice time series containing a sequence of altitude profiles of data.

Example profiles from the EISCAT Svalbard radar: Log10 electron density, electron temperature, ion temperature, ion velocity component along beam

Example EISCAT radar profiles.

Just like that. Now, the radar produces a lot of these profiles, for example one per minute. What’s the best way to organize these data? That’s right, in matrices where each column is one profile. And then you just stack them in sequential order. That way it’s easy to make scripts that snip out just the time period you want to poke at.

Below is a graph of the time axis in one version of a dataset I’m analysing. Can you spot the error? X axis profile number, Y axis timestamp of profile.

Plot of an interleaved time axis, making time series difficult to handle

Please don't do this.

This entry was posted in Gah!, The wonderful world of science and tagged , , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>