I missed the beginning of this, so here’s just some notes and braindumps.

for $wq qw wq qw {
  print $wq;
}

…prints:

q
q

You can do things like @list = <Damian Matthew Lucifer Conway> and have it just work.

sub RESPECT { print "@_\n" }
(R.E.S.P.E.C.T)->('find out what it means to me');

Oh god.

sub *{'='} = sub { print "@_\n" };

or

*= = sub { print "@_\n" }

Damian is an evil evil man. You can change the name of anonymous subroutines:

my $anon1 = sub {
  local *__ANON__ = "Bruce";
  carp "We'll just call you Bruce, okay?";
};

That’s the only bit of useful stuff in this talk. Here comes some more useless scary stuff: while ( $count --> 0 ) {. $hash<-{$key} is a substitute for an empty string.

You can slurp a whole file in a single line: my $content = readline!open!((*{$!},$/)=\$file); — I think that’s right.

And that’s a wrap!

[tags]oscon, oscon08, perl, damian conway[/tags]

I was originally going to go to Greening the Conference Circuit because these conferences are environmentally unfriendly, but decided to go to something tangentially-related to what I do. Satellite science is kind of close to astronomy, I suppose… There’s another astronomy-ish talk in the next slot that I’m not sure if I’m going to go to or not. If this one turns into an ADASS-like presentation, I’m not going to the next astronomy talk.

Curt Tilmes works at the NASA Goddard Space Flight Center. He hasn’t started yet, but he’s showing us nasaimages.org and the Earth Observatory. Tip for presenters: show people pretty pictures off the top like Curt is doing.

Acronym: EOSDIS is Earth Observing System Data Information System. EOS is Earth Observing System. MODIS is Moderate Resolution Imaging Spectroradiometer.

3.2TB per day of archive growth, total archive volume is 4.9PB. Distribute 4.2TB/day to end users. MODIS orbits 14 times a day. Spits out a lot of data to level 2 processes, which get chopped up into level 3 products (tile basis, smaller, takes out clouds if it can).

Ooo flowchart, yes this is an ADASS-style talk.

Ozone processing is done with OMIDAPS, thirty years of data.

Core system was designed in the 90s. Modular system to allow for many different projects. Used latest & greatest design methodologies and languages (DCE, C++). Requirements developed years ahead of system deployment. Many instances were planned, and several are still running very successfully today. to save money the decided to use commercial off-the-shelf software: Sybase, Tivoli, HP OpenView, Autosys… MODIS design team decided to come up with the “MODIS Emergency Backup System” built out of Perl and Linux and other things like this that they could use to process the data, just in case the big system was late.

They stuck with Sybase, but went for Apache, Linux, Perl… 80-processor Origin 2000 with 40GB of RAM, 1TB disk (in 1998/1999). In 2002/2002 they moved off a big machine to a bunch of two-processor Linux hosts, added 35TB for forward processing and 28TB for reprocessing. Now they have about a petabyte of disk, tapes are phased out…

They’ve moved quite a lot to open source: SGI to Linux, Sybase to PostgreSQL, C++ to Perl, Apache to Apache/mod_perl/Mason, ClearCase to CVS (and later subversion), DDTS to Bugzilla.

At this point I had to go out to pee, and when I got back I got interviewed by O’Reilly News again. Alas.

[tags]oscon, oscon08, nasa[/tags]

The final day of OSCON 2008 is upon us, and it kicks off with a keynote address by Benjamin Mako Hill from MIT Center for Civic Media titled “Advocating Software Freedom by Revealing Errors“. If we’re talking about control over technology, we must first understand that technology is powerful. Developers of technology have a huge amount of power. This gives them a profound control over users. Developers understand this, but most users don’t understand that that well because they don’t understand the technology as well.

Good technology should be invisible, like eyeglasses. You don’t look at the eyeglasses, you look at the world. This is why people are surprised when an ATM crashes and they see a Windows error interface — the operating system is normally invisible to them.

OneNewsNow has a hidden intermediary that is only visible when it gets things wrong: witness when Tyson Gay set a US record in the 100-meter dash. ONN has a script that replaces certain words for others — in this case, replacing ‘gay’ with ‘homosexual’. Thus their AP feed contained a story about Tyson Homosexual winning the 100.

Check out the Revealing Errors Blog.

Next comes Dawn Nafus of Intel talking about “Three Challenges“. There are three areas of rapid social change where there are real contributions that technology as a whole can make.

Challenge one: The world is awash with large datasets. How do we give life to them? Answer: avoid the ‘add GPS and stir’ trap. There’s this misconception that adding data is the same thing as adding context.

Challenge two: Global crisis in food and water. Fuel prices have skyrocketed, food prices have skyrocketed, food and water supplies in developing and third world countries are no longer secure. 50% of the world derives its income from agriculture.

Challenge three: Strength the global growth in technology producers, not just consumers. Technology production is unequally distributed. Technology adoption (in relation to economic strength) is taking off in eastern Europe, China, South America, and parts of Africa.

After Dawn is Sam Ramji from Microsoft with a talk titled “Open Source Heroes“. Well, the title doesn’t seem to be right, he’s talking about Microsoft’s future with open source. Kicking off with a history of platform trends — started off with applications running on single machines, then the internet came along, and then along came web apps. In the future, we’ll see software+services, the programmable internet. Open Source will continue to be there as well.

Microsoft has had their fingers in many bits of Open Source projects, from Linux and Apache to Ruby and Java. They submitted their license to the OSI. This section really looks like Microsoft trying to say how good they are with Open Source.

OH BOY BIG ANNOUNCEMENT: Apache Software Foundation is welcoming Microsoft as a platinum sponsor.

A few promises: “I will engage openly and honestly with you every time.” “I’m dedicated to having the tough conversations.” “We will solve the next generation of technology challenges by participating and contributing together.”

Tim Bray from Sun is up now, talking about a “Language Inflection Point“. Whare we programming in? C, Java, C++, PHP, Visual Basic, Perl… Should you do things in Perl, Ruby, or Python? In C++? (we hates it, it makes our eyes burn) In PHP? PHP has some interesting trends, and there are a number of good apps written in PHP (WordPress, MediaWiki, Drupal). “I would rather drive nails in my head than write a real application in PHP.”

Does language design matter? PHP doesn’t have good language design, but it works.

Should you do things in Java? “A lot of what is happening in languages now is unpausing the nuclear winter that Java imposed on the programming language space.” – Ted Leung

Tim is starting to have real problems with Java. “I don’t really have time to write thirteen lines of code to have four that do anything.” One of the legs of the Java tripod (JVM, APIs, language) can be replaced with other languages: Ruby, JavaScript, PHP and others.

Should you do things in C? It’s old, it doesn’t have new features like newer languages, but Tim says “C is flawless”. There are all kinds of things written in C: operating systems, databases, other languages, web infrastructure…

But what about other languages like OCaml, Erlang, Groofy, Fan, or Scala? And what about static or dynamic typing? There are good things that static typing gets you, but there are also good arguments for dynamic typing.

And last up is Jeremy Ruston from BT Design giving a talk titled “Learning From Airports“. BT wanted to learn about the innovation happening at the edges of the open source community, so they acquired TiddlyWiki, which is the project Jeremy worked on before working at BT.

There are a lot of things at airports that don’t have anything to do with the actual taking off and landing — shopping, restaurants, hospitals… Airports have single sign-on (boarding passes). The participants in the platform are trusting the platform itself for identification. Gate systems are standardized to allow all planes. Airports are very consistent in their signage.

Jeremy tried to apply this airport analogy to BT. He talked a bit but didn’t actually explain this, other than saying that they’re transparent on osmosoft.com.

Edit: The Q&A is a “beat up the Microsoft guy” session. First question: Will Microsoft stop beating up on open source re: software patents? Second question: Samba was around for at least twelve years before Microsoft helped — can we expect this kind of behaviour in the future? Third question: OOXML standards debacle where MS used their power to buy national standards community to push through an inappropriate standard — can the open source people within Microsoft change the corporate culture? Fourth question: something about patents. Blah blah blah.

[tags]oscon, oscon08, mit, intel, microsoft, sun, bt design[/tags]

The title of Larry’s talk is “Rules that are meant to be broken”. He says that he’s breaking his rule of giving funny talks, and there’s only one joke in the whole talk, and that wasn’t it.

He wants us to break Perl by coming up with new syntax for it.

People keep asking him “if you had to do it all over again, what would you change?” Two answers: nothing and everything. Perl 6 is the “everything” answer.

In Perl 6 regexes are a sub-language, not strings (as they are in Perl 5). Perl 6′s goal is to be a collection of many languages.

But in Perl culture we’re not told how to think. After all, TMTOWTDI.

Larry proposes that we deal with derived languages sensibly. Instead of copy-and-paste, use polymorphism.

I think I’m losing the plot here, so I’m going to pay attention to the talk now. Sorry!

[tags]oscon, oscon08, perl, larry wall[/tags]

Lightning talks are five-minute presentations about virtually anything related to Perl. They’re supposed to run for an hour and a half, so there should be plenty of speakers and plenty of topics.

First up is David Wheeler talking about testing your database with pgTAP, which is a bunch of stuff that is pretty much Test::More for PostgreSQL. Perl tests and SQL tests all in one bundle.

Next up is Eric Wilhelm with the 2008 The Perl Foundation Summer of Code update. There’s a lack of Perl in universities — 40 mentors, only 12 proposals, and 6 funded slots. The number of proposals needs to increase. This year’s projects are: flesh out the Perl 6 test suite (Adrian Kreher), wxCPANPLUS (Samuel Tyler), Native Call Interface Signatures and Stubs Generation for Parrot (Kevin Tew), Incremental Tricolor Garbage Collector (Andrew Whitworth), Math::GSL (Thierry Moisan), and… uh, did I miss the sixth?

Next up is Fred Meyers who works at USA Today, talking about what would be missing from the newspaper if Perl stopped working: bar code, news snippet for each state, AP “results at a glance” for election info, best selling books info, TV Nielsen ratings, week’s top stocks, stock tables, sports standings and results, upcoming games, box scores, team stats.

Next up is Michael Schwern, talking about “Who’s Afraid of 2038?” Remember Y2K? Total mass hysteria that didn’t happen. Now we’re home free. Right? Nope. “Critical and Significant Dates” In 2038 the Unix epoch is going to end — using 32-bit signed double floating point numbers. To fix this in Perl, Schwern suggested that there’s no reason for a Perl programmer should care what the underlying system uses. Schwern can solve the 2038 problem with code at svn://svn.schwern.org/svn/y2038

Next up is Brad Fitzpatrick who works at Google. He’s talking about the Google App Engine and how he’s going to get Perl implemented on it.

After Brad comes Paul Fenwick to talk about autodie. Error handling in Perl sucks. Typical error handling code (or die ...) is longer than your code. Wouldn’t it be nice if we can get Perl to generate errors automatically if e.g. your open failed? There’s the Fatal module, but it’s ugly and horrible. autodie is pretty much the same, but it’s lexical and it gives you better error messages. You can put use autodie inside an eval and then use a given/which to figure out which statement inside the eval failed. It can be subclassed too!

Now comes some guy who didn’t say who he is, talking about Net::SMTP::ESMTP, a modern email client.

Brook Wilcox is talking about a module called Devel::REPL and Continuity, which kind of gives you a Perl shell in a browser.

Jerry Gay is going to write a Turing-complete language in five minutes using Parrot. And he did! Well, he didn’t type it all live as that would’ve taken too long…

And now someone else is going to wrap this language into an Apache module — in five minutes. Live demo! And it culminated with the first LOLCAT sighting of OSCON.

Now comes Tatsuhiko Miyagawa talking about Web::Scraper. Lots of screen-scraping going on, mostly with LWP and regexps. Bad! Fragile, hard to find bugs, unmaintainable. Use Web::Scraper instead. DSL-ish syntax, CSS selectors and XPath, DOM and Unicode handling. Comes with a command-line interface. Looks really cool and useful.

Chia-liang Kaw is up talking about the Secret of Success of Open Source: plagiarize. An homage to Tom Lehrer. Well done!

Now comes the swearing portion of our show: “Fuck, the Fucking thing is Fucked” by Jos Boumans. RIPE is one of the five Regional Internet Registries. Jos Boumans works for RIPE, in case you’re wondering about the connectin. IPv4 is finite and is running out of space: IPv4 D-Day is 22/05/2010. So deploy IPv6 then! That was the plan, but plans don’t match reality. But nobody wants to move to IPv6 because nobody is using IPv6.

Now Elizabeth Cortell with “Pulp Perl” — O’Reilly’s other publishing line. ORA is famous for “anvil books” that don’t really rake in the cash. Pulp Perl is the most fun you can have reading with two hands. (spoof on pulp fiction novels and magazines with a Perl theme)

Adam Kennedy is up next, and he’s from the future (Australia). Adam likes first principles. About five years ago he wanted to find out if he could find a way to make Hollywood movies suck less? Since he can’t change Hollywood, he had to change himself. He’s going too fast so I can’t type to follow along. Alas. Oh, takeaway statement: do not watch trailers.

And now pudge with a singalong. Woo!

[tags]oscon, oscon08, perl[/tags]

Why log? Logging can help you debug during development, go back in time and figure out what happened, measure performance, and to trace activity on live systems.

So why use Log4perl? It doesn’t have any CPAN dependencies. It’s easy to use, but scales with complexity. And it has some unique features.

There are four different kinds of remote controls: levels (log/suppress priority/level), loggers (locate it in the system), layouts (format it), and appenders (write it out).

In your code, you can set different levels by doing things like:

DEBUG "Starting up";
ERROR "Cannot open file $file";

The output layout can be modified:

Log::Log4perl->easy_init( {
  level => $DEBUG,
  layout => "%r %M %F-$L>%m%n",
});

If you’re doing more complex things like this, Schilli recommends using a config file: Log::Log4perl->init( "l4p.conf" );. The benefits of config files is that you can modify them independently from the program, modify it while the program is running, and it can be modified by people who don’t know (or can’t access) the code.

You can set Log4perl up so it looks for a specific configuration file. Not only that, you can set it up to reload that configuration file at any given frequency, which allows your logging to be modified while your program is running.

The available appenders are: Screen, File, Socket, DBI, Synchronized, RRDs, and a bunch more supplied by Log::Dispatch. You can set up rotating logfiles via Log::Dispatch::LogRotate.

Layouts support all kinds of different information, and if there’s a layout pattern that you want that’s not available, you can define new ones.

Log4perl isn’t “free” in the speed sense: 50k logged messages per second gives you some overhead in your program.

Wouldn’t it be nice if more CPAN modules were Log4perl-enabled?

[tags]oscon, oscon08, perl, log4perl[/tags]

Tim Bunce is the author of the recently-released NYTProf v2.

I unfortunately missed the first five minutes of this seminar, as I was tied up giving an interview for O’Reilly News with chromatic. Tim, chromatic says hi and thank you for the help with XS.

NYTProf v2 does subroutine profiling, meaning it is a statement and a subroutine profiler, at the same time. The subroutine profile tracks time per calling location. It calculates exclusive time on-the-fly, and discounts the overhead of statement profile. To minimize disk I/O, it does everything in memory and writes to a file at the end. It’s also extremely fast. Statement profiler gives correct timing after leave operations.

v2 does sub microsecond resolution. It reports correct timing for mod_perl, handles forks with no overhead. Profiling can be enabled and disabled on the fly, and can profile compile-time activity.

When Tim asked for questions, Adam Kennedy asked, “The only question is, ‘Why are you so awesome?’”

The plan is to merge multiple runs, so say from running code through a test suite.

Reporting is only done via HTML or a CSV file. Tim isn’t too happy with this, thinking that he could come up with some pluggable class structure.

Trace file size depends on how long the profiler runs on your program. For example, running perlcritic on the NYTProf source code created a 360MB to profile 26 million statements. This ran for a minute and a half (which normally takes 30 seconds).

[tags]oscon, oscon08, perl, tim bunce, nytprof[/tags]

The full title of this talk is “Skimmable Code: Fast to Read, Safe to Change”, and it’s presented by Michael Schwern.

What does Schwern mean by “skimmable”? He means in terms of like skimming a book — you skim through the book to find info you want, rather than studying the entire thing, because you want to find the info fast.

Why skim? It allows you to focus on a single task. It lets you slice through hairy code. It lets you free up brainpower. This talk is essentially applied best practices, it explains the reasoning behind best practices.

How do you skim? It all comes down to narrowing your focus. How do you do that? Your primary weapon is “lexical encapsulation” — look at the stuff that’s right in front of your face. Not the subroutine calls, not method calls, just the stuff between the braces (that’s in your lexical scope).

Tests are useful for making code skimmable, because at the touch of a button you can see if your modifications have broken anything. When refactoring code, refactor, then test, then commit, then repeat. If you fail a test, you can find out where you’ve broken things (and always test before committing).

Rename refactoring: use good names. For example, calling something $arg isn’t useful; call it something that describes what it is actually used for. This way you don’t have to keep referring back up to the top to figure out what the argument is.

Extract refactoring: extract routines and methods out to prevent other routines from doing too much that you can’t keep it all in your head. After you extract routines out to subroutines, it cleans up the structure of your program and you may be able to extract more routines out to subroutines again.

Whitespace allows clarity. Vertical whitespace can be used to differentiate between major blocks in your code. Similar things should look similar, so if you’re using a hash constructor try lining up the fat commas.

Different things should be displayed differently, e.g. $lexical vs $Global.

Red flags: No documentation. No tests. Things are deeply nested. Compound names (e.g. make_waffles_and_pancakes. Generic names (e.g. @list, $arg, $data). Overly verbose (e.g. $total_amount_spent_on_food). Globals. Scope closing comments. Long routines.

[tags]oscon, oscon08[/tags]

We kick things off with Keith Bergelt talking about “Open Invention Network and Its Role in Open Source and Linux”. We’re just part of a larger group that exists to tackle current realities and problems with intellectual property and open source. One of the realities is that open source is the single-most important invention and innovation modality to come into being in the post-industrial era. (Oh no he just used the word “utilize”. Oh no “paradigm”.)

<CosmicRay> is this one of the talks that was randomly generated by that conference talk generator that was on /. awhile back?

The reality is that there’s a growing number of patent trolls, and people are turning this into a business model. Defensive patent pools are growing. Rabble rabble business speak rabble rabble.

Next up is Peter H. Salus talking about “Anniversaries”. Please don’t be as boring as Keith Bergelt, please! This looks to be an historical talk about software and whatnot. We talk about anniversaries and make the call that anniversaries are special when they’re divisible by 5 or 10. For example, this year is the 250th anniversary of Darwin’s birth. Last December was the 60th anniversary of the transistor. In the same year, IBM installed the first SSEC — selective sequence electronic calculator — the first electronic computer, in the modern sense of the word. A decade later DEC started with venture capital of $40,000. In 1963 Project Mac was formed at MIT — and out of that came computer sharing. Ten years after that the first Unix paper was written by K&R.

Next year is going to be a great one for anniversaries: moon landing, Unix, ARPANET with four nodes — expanded to 216 in eight years and they were worried about hitting 256 and running out of address space. “If you pay attention to history, all of the stories are old stories. The magnitude may change, but they’re all the same stories.” First free software meeting was in 1996.

He’s coming out with a book soon about the history of open source, which should cover most of what he’s said in this keynote.

After Peter comes David Recordon of Six Apart with a talk titled “Supporting the Open Web”. We’ve been talking a lot about open source, and there’s a lot of open source to talk about. Open Data is increasingly important as services move apart. The open web needs open data, and open data needs open specifications. The web needs to be accessible everywhere. In 2007, the web started becoming more open. OpenID, OAuth, and Open Social are all quite similar: they have communities ranging from individuals to companies, implemented in many languages, being adopted at an increasing rate, and are occurring outside of formal standards bodies. The Open Web Foundation is announced so companies and individuals can come together to work on the open web. It will focus on four things: incubation (creating new open specifications for the web), licensing (or really no licensing; non-assertion agreements), copyright (Creative Commons for each specification), and community (to support the open web).

Danese Cooper comes next with her presentation about how “Why Whinging Doesn’t Work”. She kicked things off with the Helsinki Complaints Choir. Why do we love to whinge? There are a lot of theories: misery loves company, we do it because we’re fundamentally afraid. Confirmation bias is “what you believe comes back to you” — if you believe that the glass is half-empty, then you’re going to see half-empty glasses wherever you look.

What does complaining do to us? It makes us isolated. You need to think differently — Billie Jean King pulled her competitors together to rally for equal pay for female tennis players as male tennis players. We need to recalibrate our language. We need to radiate gratitude — i.e. Randy Pausch. We need to acknowledge others. We need to be the world we want to live in.

Yay! Nat is back! Nat’s talk is titled “fork() && exec(): Spawning the Next Generation of Hackers” which sounds awesome. No aloha shirt though. :( This talk is not about how to have sex. This talk is about raising children to use computers. When Nat moved to New Zealand, he found that the primary school had utter crap for computers, and the administrator had the best one in the school. Nat ran Kiwi Foo and took some of the leftover money and bought the school a nine new computers. Then who could teach the kids how to use them? The teachers couldn’t because they had crappy computers and didn’t have the right skills, so Nat did. He had to start off with basic computer skills, but he wanted to move to teaching them how to program.

Programming is like changing a tire. It’s not rocket science. Nat kicked things off by running a computer club. He started with Lego Mindstorms, but the kids found them lame. You can make them go forwards, backwards, turn, and follow a line on a page. But they don’t, so robots aren’t that great for kids who need consistency and get disappointed when the Mindstorm goes off on some random tangent. scratch is much better for kids. With his own kids, once they progressed past scratch he moved to processing, which is more complex.

Lessons: lectures suck. You have two minutes tops when you stand up in front of kids. The gender gap is not what you think (at the eight-year old level). You have to follow their interests. Keyboards are a challenge. Math skills aren’t there yet (angles, e.g.). Teachers need teaching. Robots = lame.

An immodest proposal: 1) Volunteer in schools. 2) Courseware. Look at scratch, build your own courseware and share it.

Further reading:

• OSCON 2008 keynotes – Thursday morning via samwise
• OSCON 2008: Thursday Morning Keynotes via sirhc.us maxim.us

[tags]oscon, oscon08[/tags]

Our story begins in ancient greece, where they created a device that could compute the positions of the planets, the sun, and the moon: the world’s oldest computer. It was plundered by the Romans, and on the way to Italy the boat sunk, and was recovered in 2001.

You know, I’m not going to blog this. You suckers who aren’t here are out of luck, because Paul Fenwick is an entertaining presenter.

[tags]oscon, oscon08[/tags]