canspice.org

I was originally going to go to Greening the Conference Circuit because these conferences are environmentally unfriendly, but decided to go to something tangentially-related to what I do. Satellite science is kind of close to astronomy, I suppose… There’s another astronomy-ish talk in the next slot that I’m not sure if I’m going to go to or not. If this one turns into an ADASS-like presentation, I’m not going to the next astronomy talk.

Curt Tilmes works at the NASA Goddard Space Flight Center. He hasn’t started yet, but he’s showing us nasaimages.org and the Earth Observatory. Tip for presenters: show people pretty pictures off the top like Curt is doing.

Acronym: EOSDIS is Earth Observing System Data Information System. EOS is Earth Observing System. MODIS is Moderate Resolution Imaging Spectroradiometer.

3.2TB per day of archive growth, total archive volume is 4.9PB. Distribute 4.2TB/day to end users. MODIS orbits 14 times a day. Spits out a lot of data to level 2 processes, which get chopped up into level 3 products (tile basis, smaller, takes out clouds if it can).

Ooo flowchart, yes this is an ADASS-style talk.

Ozone processing is done with OMIDAPS, thirty years of data.

Core system was designed in the 90s. Modular system to allow for many different projects. Used latest & greatest design methodologies and languages (DCE, C++). Requirements developed years ahead of system deployment. Many instances were planned, and several are still running very successfully today. to save money the decided to use commercial off-the-shelf software: Sybase, Tivoli, HP OpenView, Autosys… MODIS design team decided to come up with the “MODIS Emergency Backup System” built out of Perl and Linux and other things like this that they could use to process the data, just in case the big system was late.

They stuck with Sybase, but went for Apache, Linux, Perl… 80-processor Origin 2000 with 40GB of RAM, 1TB disk (in 1998/1999). In 2002/2002 they moved off a big machine to a bunch of two-processor Linux hosts, added 35TB for forward processing and 28TB for reprocessing. Now they have about a petabyte of disk, tapes are phased out…

They’ve moved quite a lot to open source: SGI to Linux, Sybase to PostgreSQL, C++ to Perl, Apache to Apache/mod_perl/Mason, ClearCase to CVS (and later subversion), DDTS to Bugzilla.

At this point I had to go out to pee, and when I got back I got interviewed by O’Reilly News again. Alas.