What does “postmodern” mean? It comes from a talk Larry gave back in c.1999, where the best bits are pulled from all kinds of different sources. This is done in Moose, but it pulls from other object systems: Ruby, smalltalk, Perl6, Bethe (sp?), and a few more.

What is Moose? To answer that, we need to answer “What is Class::MOP?” To answer that, we need to answer “What is a MOP?” MOP is Meta Object Protocol, which was stolen (for Moose) from CLOS. A MOP is a formalization of the object system. It’s a way to describe how classes and methods work, how objects are created, etc. Class::MOP is CLOS for Perl 5. Class::Mop is designed to be a platform for Moose.

This is the simplest Moose code:

package Person;
use Moose;
has name => ( is => 'rw' );
has age => ( is => 'rw' );
1;

It turns on strict and warnings. The Moose::Object is pushed into @ISA for you. You get a lot of class-level metadata as well.

Moose has types, in the Haskell deep type system. You can do something like:

has name => (
  is => 'rw',
  isa => 'Str',
  required => 1
);

Or add a default using default => sub { DateTime::Duration->new }. Or use another object type: isa => 'DateTime::Duration'. Thus validation is done, default values are done…

You can create custom types using Moose::Util::TypeConstraints:

subtype 'NonEmptyStr'
  => as 'Str'
  => where { length $_ > 0 }

Then you can use these types just like the pre-existing types (like Str).

As an example of creating one of the Person objects:

my $me = Person->new(
  name => 'Stevan',
  age => DateTime::Duration->new( years => 34 )
);

Type coercions can be used to transform different types into the types you’re looking for, like changing an Int into a DateTime::Duration object.

Delegation can be used to “pass off” a method to a method of the Moose object type. Kind of like inheritance. If you “delegate” the years_old method to years, when you call the years_old method on the Moose object (which is actually a DateTime::Duration object), then it calls the years method of the DateTime::Duration object.

Moose meta-attributes can be overridden via extensions. Method modifiers are available, which allow you to do things like run code after a method has been run.

General rule of thumb: roles should be used when you go reaching for multiple inheritance, think about roles instead. Don’t use roles when a class just makes more sense. Roles are not a replacement for inheritance.

[tags]oscon, oscon08, perl, moose, object oriented programming[/tags]

Theo Schlossnagle leads the performance and scalability practice group at OmniTI. He “un”breaks things, and builds high-performance software.

What does full-stack mean? This means looking at the entire stack of operating system, software, data storage infrastructure, middleware, etc. How do you see them all at the same time? Using a debugger you know how to look into your program. But how do you see what Apache is doing? How do you see what the DB is doing? The frustrating part is that it’s difficult to interleave what you’re looking at between the different systems.

DTrace can look at the whole stack. It doesn’t run on Linux (yet) but does on Solaris, OS X, FreeBSD. It looks at Apache. For DBs, it looks at MySQL and PostgreSQL. For languages it looks at PHP, Perl, Java, Ruby, C, C++, and anything compiled with frames.

DTrace allows you to write scripts that can consume output from various traceable points in a running system, such as kernel exit and entry points, the IP stack, the system scheduler, I/O events, etc. It also allows you to put arbitrary points in user-land code (entry/return/op codes) and any machine code inside them.

The quality of answers DTrace provides is limited by the quality of questions it is asked. The syntax is a mixture of awk and C. Concept is simple: you watch stuff. Implementation is safe. Learning to ask the right questions is difficult, so use the scientific metnod. Be rigorous and analytical. Use repeatable tests, and don’t change more than one thing at a time. Use experience to draw conclusions.

There are three approaches: start closest to the user (safest), start at the problem and solve it (sorcery), or start in the middle (look for symptoms, form a targeted hypothesis, then solve the problem).

At this point Schlossnagle gave demos on using DTrace to solve real problems (instead of contrived problems).

dtrace -l gives you the list of probes available on the system. dtrace -P syscall -l will give you all of the syscall probes.

dtrace -q -n 'syscall:::entry{@[provefunc] = count();}' gives something, I know not what.

dtrace -n 'postgresql*:::statement-start' will trace PostgreSQL statment starts. You can then print off the SQL queries in real-time through some other arcane command.

[tags]oscon, oscon08, dtrace[/tags]

The full title of this session is “Creating and Supporting Free Software in Africa: the African Virtual Open Initiatives and Resources (AVOIR) experience”. Prof Derek Keats is the Executive Director of Information & Communication Services at the University of the Westeren Cape Bellville (Cape Town), South Africa.

Most of the world is not like Stanford. The amount of tertiary education in Africa is tiny compared to the US, Europe, and Japan. Scientific output is even more dramatic, and the number of internet users is just as dramatic. HIV/AIDS and malaria affect Africa way more than in other continents. This presentation is much more than software — the majority of people in Cape Town live in shantytowns.

The scenario is there is a lack of critical mass to find knowledge-based solutions to African challenges. There are fewer actively researching Computer Science (and most disciplines) professors in Africa than there are in a typical middle-sized American university. How do we solve this? Through collaboration. There isn’t really any other way.

The vision is to establish a pan-African F/OSS ecosystem so that African IT professionals can respond with local solutions to the needs of African society while also tapping global business opportunities. AVOIR was established as a network for capacity building in F/OSS engineering. They’ve been around for roughly two and a half years.

AVOIR is a network of universities, an alliance of partners. They’ve developed collaborative processes and tools, and an application framework called Chisimba. Applications are based on Chisimba, and they’ve set up supporting technologies and applications (mailing lists, etc), and are beginning to be sustainable.

R&D partners are located in US, Canada, Mauritania, and Afghanistan, and supporting partners are in the US. The alliance is a network of networks — increasing the number of networks in the alliance strengthens both the alliance and the underlying networks. A training workship in Kabul was run and trained a number of people — looks like from the picture it was ~65% women, which is outstanding for Afghanistan. They’ve also sent some students to UWC. They’ve done training for the SADC parliamentary forum, training IT staff. And training has been done in the Philippines at public universities there.

Four steps in the software development process: Connecting, Collaborating, Implementing, and Coding.

Chisimba is a framework for “building software and building skills.” It’s an object-oriented framework for rapid development of distributed applications. It’s also used for learning good software engineering principles and practices. Learning developers can build confidence through collaboration. It’s built on the typical suspects: Linux, MySQL, PEAR, PHP, etc.

Updates, modifications, etc., are all done through a web browser. It’s very open as well, it’s easy to bring data in (i.e. from YouTube, flickr, etc) or send data out (embed a real-time collaboration plugin in your WordPress blog).

Institutions implement AVOIR products to enable their own institutional processes. For example, eLearning at UWD, NOLNET in Namibia, electronic thesis and dissertation system at National University of Rwanda and UWC.

Social Content and Networking for Schools is a project to introduce some internet applications in an offline fashion to schools in poor regions, because the cost of internet connectivity is too high.

The biggest challenge is bandwidth. Institutions are also lacking in understanding of the value of networks. There’s a cultural predisposition to silence — the concept of everybody being equal on the network is not easily grasped. Salary structures in education are such that professors and teachers often have to work two or three jobs. There’s also a high turnover of students in some countries.

Very good support from SA government in the first year, then a change of people resulted in a decrease of funding. Pretty much no government support from other countries. Namibia came in after AVOIR started up, and they bring in government and international support.

One way to support AVOIR is to raise awareness. Big companies often have an ‘in’ with government, so if you can convince them to talk to governments then things will hopefully go well from there. Join the mailing lists and as a developer you can help out.

[tags]oscon, oscon08, africa, avoir[/tags]

There haven’t been any significant new features in Perl since 2002. This talk will give features that aren’t completely insane, aren’t just for the masters.

5.10 adds new operators and operators that will break when run under older versions of Perl, but they’re not enabled by default: use feature 'mtfnpy' and use 5.010.

say is a new builtin that is just like print, but it puts a newline at the end.

Truth and definedness is handled better than before:

sub record_sale {
  my ( $product, $amount ) = @_;
  $amount ||= $product->cost;
  ...
}

But what happens if the cost is zero? Then you had to use some ternary operator code. In 5.10 you can use the new “or” operator: $amount //= $product->cost;

State variables can replace a lexical or something messier, so you can do this:

sub read_line {
  state $lines_left = 100;
  die "trial period expired" unless $lines_left-- > 0;
  ...
}

The first time in, the variable gets initialized. Next time in, it doesn’t, and it keeps the value when it went out of scope.

File tests are now stackable: if( -f -w -z $file ) instead of if( -f $file and -w $file and -z $file ).

Smart matching is a new kind of comparison. It depends on the input to the operator. The smart match operator is ~~ and tries to figure out the right time of match. You can do $foo ~~ undef (check for definedness), $foo ~~ @array (it’s in the array), coderefs, in a hash (don’t know if that’s in keys or values though).

Perl 5.10 has a switch operator: given and when. It uses the smart matching to do its comparisons.

SmartMatch::Sugar does powerful smart matching.

Better error messages for “unknown undefined” values. When you do things like printing out variables and get the “use of uninitialized value” warning, it will tell you which variable is undefined.

Traditional Perl5 objects are made with a hashref. Looking up attributes means you look up the actual attribute name. In 5.10 you look up the attribute in an external hash by the object ID. This introduces a field hash:

sub size {
  my $self = shift;
  fieldhash state %size;
  if( @_ ) { return $self{$size} = shift; }
  return $self{$size};
}

Lexical $_ (he calls $_ “topic” because it’s easier to say). You can now do things like for my $_ (@lines) { ... } and when you leave scope, $_ doesn’t get changed when you might not expect it to (say if there’s a subroutine that modifies it).

Regular expression named captures: find matches by name, not by position. No more $1! Now you can do $line ~~ /(?\w+):(?\w+)/ so then you can access $name and $value instead of $1 and $2.

[tags]oscon, oscon08, perl[/tags]

“Normal” in this sense refers to database normalization. Mike Hillyer is a senior engineer at Message Systems.

What is normalization and what are its benefits?

Actually this looks to be too low a level for me, so I’m going to either hit the expo hall or find another session.

[tags]oscon, oscon08, database, normal[/tags]

Tim O’Reilly discussing history of OSCON: Started out as The Perl Conference twelve years ago. We’re apparently really crappy at keeping our history. Early conferences had a track on Tcl/Tk! Steve Jobs and Mark Shuttleworth separated at birth? Open Source related jobs are about 1 in 55 for tech companies, 1 in 385 jobs in non-tech companies. The number has grown over the years. Linux jobs up 19%, Perl down 7%, Python up 56% (but sixth overall).

Three big challenges and opportunities: cloud computing, the (open) programmable web, and open mobile. Cloud computing means we’re subject to other people’s failures.

Also, “web” doesn’t mean http, it also means things like XMPP. Today it even means “phone”.

Programmable web: data is becoming a new source of value and lock-in. We need to think about this. e.g. CDDB, Kindle. Because of competition, people are pushing in the other direction: Yahoo! BOSS service, Dopplr…

Open mobile: the browser wars are back. iPhone app store is drawing a lot of developers. “Google understands that if the mobile phone isn’t open, they’re toast.” They understand how much they depend on the open source ecosystem. And the OS community is there: OpenMoko, Android.

Christine Peterson on Open Source Physical Security. “Problems that start in the US tend to spill over and hurt people elsewhere.” The future, it’s not just electronic, it’s also physical. Nanotech-based sensors are easily programmable. They’re already being used in sewers. They can be abused, though. Things worth detecting: WMDs. But fear + poor data on WMDs leads to bad things. Result: more surveillance. DC only has top-down tools, and they don’t notice our debates. They’re trying to solve a bottom-up problem (terrorism) with top-down tools. Rabble rabble nerds are the best to fix this rabble rabble.

Dirk Hohndel on Moblin, Linux for Next Generation Mobile Internet. The vision is about the internet, mobility, flexibility, and extensibility. We need an Eee PC that doesn’t have a $2000 price tag, it needs to be more $300-500. The fear of lock-in to a platform is greater to many people than data lock-in. A year ago Intel started Moblin and invited other people to join them. Unfortunately most people that joined were looking to make a buck, not many open source people came in. They’re looking for more open source people to come in.

And then something about MySQL happened, and I went to the expo hall.

[tags]oscon, oscon08, oreilly, tim oreilly, moblin, security[/tags]

I received my OSCON 2008 registration package yesterday, and as with past years I received a bag filled with useless crap. There’s a foam rocket, there’s some bizarre squishy thing, there’s a cheap plastic pen, and there’s the usual assortment of inserts, advertisements, magazines, and other crap that I’m not going to use or look at.

The kicker is a big glossy magazine from Web Host Industry Review. That wouldn’t be so bad, but it’s the “Green Edition”. Fuck you, a glossy magazine that’s going to get thrown in the garbage by 90% of people receiving it is not green.

Why can’t O’Reilly put all this information on a USB flash drive and give those to people? Flash drives are cheap, you can cram all kinds of information on them, and they’re useful! If I don’t want the free copy of The Linux Journal, I don’t have to throw it away (or best option, recycle it), I just delete it from the flash drive. And then I have something useful to take away from the conference!

So come on O’Reilly, start using USB flash drives!

[tags]oscon, oscon08, environment, recycle, web host industry review, oreilly[/tags]

People For Geeks, the Tuesday afternoon tutorial I’m attending, is being presented by Michael Schwern, Selena Deckelmann, Brian Fitzpatrick, Ben Collins-Sussman, Andy Lester, and Kirrily Robert.

First thing, from an editorial point of view, I typically don’t like these types of tutorials because the speakers don’t all have microphones and you end up missing what people say, or they’re passing around one microphone between everybody and it kills the energy and synergy you get between a bunch of speakers. However, it looks like they’re just going to be talking one at a time, giving a pack of mini-tutorials.

Anyways, to the tutorial. Slides will be on Slideshare, tagged with ‘peopleforgeeks’.

Kirrily Robert (aka Skud) is up first, talking about geek etiquette. Cell phones off, try not to pay too close attention to your laptops, please. Ptah-Hotep wrote the first etiquette book, and it’s surprisingly accessible. One of the first modern etiquette authors was Castiglione, who started the idea of a Renaissance Man. And one of the big populizers of etiquette was William Caxton. One of the most popular purveyors of etiquette is Emily Post.

Why spend any time worrying about etiquette? There’s a group benefit to it. When interacting with other people, etiquette benefits the other people. It helps the group function better together. There’s also a personal benefit to it. People are going to judge you on how you behave, and if you behave well, they’ll judge you positively. And some people say that behaving well towards other people is some sort of fundamental ethic. Major religions have some sort of Golden Rule; it’s almost universal across religions, philosophies, etc.

So how can we be good to other people? That’s what all the etiquette books are about, but there are so many rules that it’s hard to tell which ones are actually important. There’s an 80/20 rule that says that in many situations, 80% of the benefit is achieved by doing 20% of the work, and vice versa. Here are the rules that make up the 20% of the work: respect, listen, and shower daily. Mostly, it’s not rocket science.

Schwern is up next, talking about how not to be a jackass. So, why shouldn’t you be a jackass? Some people think that being a jackass is fun. It is, but only for you. Some people think that being a jackass makes you feel superior. Are you really, though? Ultimately most reasons for being a jackass are selfish and don’t get you anywhere. If you know someone’s going to misinterpret what you’re saying, you’re doing them a disservice (you’re “lying by jackholery”). There’s a mode Schwern calls “Literal Fuckhead Mode” that (obviously) isn’t nice either.

Oblique Strategies For Miscommunications: Say “please” and “thank you”. Apply tact filters to your output. Not everything that comes to mind needs to come out the mouth. Stop and count to five. Are you just repeating louder? Bitching is not doing (bug reports are doing). Start the solution. Why does it matter to you? Remember the goal. Are you having fun? How do you both win? There is no best. Don’t get even. Give an out. Praise in public; critique in private. Acknowledge the problem. Is the problem a bad solution? Give them power to solve their problem. Don’t be afraid to say “I don’t know.” “Tell me about that.” Shut up and listen. Don’t claim to be an expert. Talk about them. What makes them happy? Find the lost information.

Next up is Andy Lester, talking about “how to speak Manager”. Andy’s started a blog about geeks in the workplace. There are many reasons to speak Manager, and many of them benefit you.

Principles to think about: Your relationship with your boss is a relationship. If you’re doing a regular 9-to-5 job, you’re spending more waking hours with co-workers than you are with your spouse. Your boss is not an enemy, nor is he incompetent. If any of these are false, you need a new job. Life is too short to work for a boss who is an enemy or who is incompetent, and it’s too short to work at any company that would allow such a boss to manage people.

How do we improve communication? It’s hard, and no one else thinks like you do. You may be the only one working on improving communication. You need to think like the boss. After all, your number one job is making the boss look good. You and your boss share one goal: you both want to get your jobs done. The less time spent with your boss, the better, because if you’re going to see your boss, you’re not working on your project.

What is unnecessary interaction for the boss? Anything that doesn’t get results, people problems, arguments, dealing with your personal problems, whining, things you can solve yourself.

What is unnecessary interaction for you? Micromanagement, anything that doesn’t let you do cool stuff.

How to speak Manager? Business speaks in two currencies: money and time. Your job is to translate the techie details to money and time. This is what you do when you work on your résum&eactue;. It is always better to give bad news about yourself than to have your boss discover it through some other means.

No competent boss has ever OKed “a week to clean up the code.” You have to explain what you want, the costs, and the benefits.

Selena Deckelmann is up now (after coffee break) with a talk titled “Leading without being in charge”. So you’ve got an idea for a user group. You’re looking for people to join. How do you do this? You want them to come to meetings, to participate, to give talks, to take over someday. What is leadership? You’re not a dictator, you can’t demand that people participate. You’re a facilitator, a moderator, and a guide. That’s what leading without being in charge is all about.

Five-step program to lead well, develop new leaders, and have enthusiastic followers:

1. get people to the meeting
2. start the meeting
3. make people feel good
4. set the tone
5. take questions

Step one: get people to the meeting. Start with your friends. Find other similar user groups and invite people from them. Use free advertising: email, blog, talk to people. Use non-free advertising: trade mags, newspapers.

Step two: start the meeting. Share the agenda before the meeting. Make announcements. Make introductions — go around the room and ask questions: why did you come today? what do you want to learn? what’s your favourite…? Introduce the speaker.

Step three: make people feel good. Rituals make people more comfortable. Remember names. Make yourself feel good. Eat and drink together.

Step four: set the tone. Be an active leader. Ask for help. This takes the load off of you and makes the other people in the organization feel like they belong and they’ll come back to help. Give people real work, not just make-work. Allow them to fail. Every user group has That Guy, he’s sitting in the front row, asking distracting questions, disrupting the flow. Deal with these people before they happen: have an agenda, save questions for last, and help the speaker out. Structure user groups for socializing, and make time for you to socialize.

Step five: take questions. In addition to taking questions, you should also ask people what they think. Then write that information down so you don’t forget it. Take their feedback, and change things (or not).

Next up is Ben Collins-Sussman and Brian W. Fitzpatrick with a talk titled “Everything That Everyone Else Said Was Wrong”. Actually it’s titled “How to Protect Your Software Project From Poisonous People”. How do you deal with people who aren’t behaving well? You first have to comprehend the threat. Attention and focus are you scarcest resources. You must protect them. Poisonous people can distract, emotionally drain your community, and cause needless infighting. You need to avoid paralysis — perfectionists and people obsessed with process are good at derailing forward progress.

How do you fortify against the threat? Build a strong community based on politeness, respect, trust, and humility. Build a mission by picking a direction and limiting your scope. Document your design decisions, your bug fixes, your mistakes, and your code changes. Have healthy code-collaboration policies: send commit emails, encourage email review, do big changes on branches for easier review, be generous with branches, increase your project’s “bus-factor” (the number of people that need to get hit by a bus before your project is screwed), don’t allow names in files. Have well-defined processes when releasing software, accepting and reviewing patches, and admitting new committers.

How do you identify poisonous people? They have communication annoyances: uses silly nicknames, overuses capital letters, uses excessive punctuation. They’re clueless: unable to pick up on the “mood”, don’t understand common goals of the community, ask incessant RTFM questions. They’re hostile: insult the status quo, angrily demand help, attempt to blackmail, make accusations of conspiracy. They don’t co-operate: willing to complain but not help fix anything, unwilling to discuss design, too insecure to take criticism.

How do you disinfect poisonous people? Assess the damage. Is this person draining attention and focus? Is this person paralyzing the project? Don’t feed the trolls. Don’t get emotional. Look for the fact under the emotion. Extract a real bug report, if possible. Know when to give up and ignore them and when to forcibly boot them from the community. Repel trolls with niceness. Address the behaviour, not the person.

[tags]oscon, oscon08, geeks, etiquette, managers, poison people, community[/tags]

Perl Worst Practices is the logical antithesis to Best Practice Perl. In this tutorial, Damian Conway will tell us about all the bad things in Perl.

Best practices lead to code that’s maintainable, robust, and efficient, by coding predictably, repeatably, and consistently by adhering to “the rules”.

Worst practice programming leads to code that’s innovative, entertaining, and fun by coding intuitively. We’re going to look at some extremely Worst Practice Code, and draw a series of positive suggestions on how to code and how not to code. We’ll see Damian’s idea of obfuscated Perl, which is a scary prospect.

SelfGOL is his obfuscated code, and it can self-replicate, rewrite other Perl programs so that they self-replicate, detect un-rewritable programs, play Conway’s “Game of Life”, and animate any text as a cycling marquee banner. And the entry is in fewer than 1000 bytes of code. If you want to see it, it’s on the Perl 5 Wiki. Actually now that I look at that and compare it with the SelfGOL in the handout book, they’re different. Oh well.

SelfGOL is full of dark backstreets, everything you wanted to know about Perl and should have been afraid to ask.

In looking at SelfGOL (or other hideous programs) first run it through perltidy. Unfortunately this breaks SelfGOL, but that’s another story.

Principle 1: Sane and consistent layout makes code more maintainable (but it isn’t a magic bullet if the code itself is beyond help).

Principle 2: Using built-in features isn’t necessarily smarter or cleaner (even though fellow developers’ futile struggles to recall those features can be highly amusing).

The next line of SelfGOL ($;=$/;) uses punctuation variables. $/ is the input record separator, and if you use it in place of the newline character you can shorten your code! $; is the hash key separator:

$hash{'x','y'} = $val;

Perl replaces the contents of $; and turns it into one key:

$hash{'x'.$;.'y'} = $val;

This whole line makes a newline “constant” that’s difficult to read.

Principle 3: Obscure and obsolete features are obscure and obsolete for a reason (and retasking them for even more obscure purposes doesn’t help).

The next line (seek+DATA,undef$/,!$s;) looks for the __DATA__ block. The IRS is cleared, which means that the next read from the DATA handle will read everything in. It also returns undef, which becomes 0 in scalar context, so seek is told to seek 0 bytes relative to the value of the -s command-line option. If -s is on the command-line, then seek starts at the beginning of the file, otherwise it starts from the current position. In other words, if -s, then seek to the start of the file, and if not -s, do nothing.

Principle 4: Each statement should do one thing only (since that’s the upper limit most brains can comprehend).

Next line ($s && print) says “was there a -s on the command-line?” If $s is set, then print the value of $_, which prints the entire stuff slurped into $_ (which was done by a skipped line: $_=<DATA>;). Voila, this is the entire bit of the code that prints the entire program.

Principle 5: Relying on default behavious makes code very slightly easier to write and vastly harder to read (because most readers can see better than they can think).

Continuing that line (|| (*{q;::\;
;}=sub{$d=$d-1?$d:$0;s;';\t#$d#;,$_}) && $g && do {) doesn’t make much sense, because most of it is used to predeclare another subroutine. The newline is important, because this actually sets up a subroutine whose name is “semicolon newline”.

Principle 6: Randomly placed subroutine definitions are static (in the radio interference sense).

Continuing ($y = ( $x ||= 20 ) * ( $y || 8 );), this sets the width ($x) and height ($y) of the board from the command-line. If $x isn’t set, then ||= will assign 20. Likewise, if $y is missing, it defaults to 8. But! $y starts with the height of the board, but internally it stores the number of cells in the board. Why would you do this? Because it suits the internal representation of the board. The obvious representation would be an array of arrays, so obviously we can’t use that. Instead we “unroll” the arrays (C programmers would find this natural) and use modulo arithmetic. The board is a 1D string storing a 2D board with a genus-one 3D topology.

Principle 7: Choose data structures that simplify your task (even if the task is to make data structures incomprehensible).

Next line (sub l { sleep &f; }) is a utility “pause” subroutine in a nested block, but it’s not scoped to that block. &f; means call f() with this sub’s @_.

Principle 8: Just because you use some operation frequently doesn’t mean it should be in a utility function.

Next line (sub'f{pop || 1}): 'f is the same as ::f (comes from Perl4). By default, pop pops @_. If no arguments, return 1. This means that &f returns $_[0].

Principle 9: Encapsulating the familiar can decrease maintainability (refactoring isn’t a substitute for sanity).

In the next line p() prints the GoL board. Start with numerous newlines to clear the screen ($= is the page length specifier, which defaults to 60). Then we need to split the board up into chunks that’s $x characters wide ($b =~ /.{$x}/g). .{$x} matches $x characters at a time, /g in list context returns a list of all matches. this means that $b is broken ever $x characters, and putting this in the second (and more) argument to the join $; means that the regular expre

Principle 10: Treat any clever one-line solution as an alarm bell (or as an antipersonnel mine with a six-month delay fuse)

Next line (sub n { substr( $b, &f % $y, 3 ) =~ tr,O,O,; }) sets up a subroutine that counts live cells in the cell’s 3-neighbourhood. &f takes the first argument, modulo-$y wraps the board at the end of the string, we grab three characters, and use tr/O/O/ to count ‘O’s.

Principle 11: Familiarity breeds comprehension (it breeds contempt too)^[1]

The next block (which I’m not going to list here) updates the GoL board. We start off with @_[~~@_]=@_;, which duplicates $_[0] into $_[1]. The ~ is the bitwise complement operator, and ~~ is a shorter version of scalar. --($f = &f) caches the first argument minus one. $m = substr( $b, &f, 1 ); grabs the current cell. Note the two &f calls — because of the way &f is implemented, the pop pops off from @_, which is why we needed the @_[~~@_]=@_ line to duplicate the first argument. The next block of code counts the number of neighbours, removing the count of the current cell which is (${m} eq +O=>). +O=> is just the letter O, because the + is a no-op and the => is the fat comma. This count is used as an index to the state table ( $w, $w, $m, O ).

Principle 12: Table-driven solutions are clean, efficient, and extensible (as long as you don’t mind losing a little comprehensibility).

At this point Chris pointed out that all of the explanations of the code are in the handout book, so now I’m just going to list the principles, some choice quotes, some little notes, and some exercises for the reader.

EftR: What does q++ do?

Principle 13: Building a messy data structure and then cleaning it up is often easier than building it cleanly in the first place (and to hell with the purists).

“It’s not a real Perl program unless there’s an eval somewhere in the first ten lines.”

Principle 14: Some code is better compiled at run-time (but the urge to use an eval is natures way of letting you know there’s not yet enough pain or misery in your life).

$i ? $b : $c = $b; can be used to modify which lvalue gets assigned to. The ternary operator has a higher precedence than assignment, so with parentheses in the right place this looks like ($i ? $b : $c ) = $b;.

Principle 15: Parentheses are our friends (because if you can remember all 24 levels of Perl’s precedence, you need to get a life).

Principle 16: Edge cases suck (and edge cases of familiar constructs suck worst of all).

$g =~ s?\d+? ($&+1)%$y ?e; might be Damian’s favourite line in SelfGOL. This modifies the actual code of the assignment in $i = 0. It finds the first digit in that string, then changes it to whatever it is, modulo $y. So when the string gets evaled again, $i will be initialized to different values. Also note the delimiters to s, the question mark. Using ? as a regex delimiter means that that regular expression will only match once between calls to the reset operator. What? Note that this only applies to matches though, not to substitutions! So in this case, ? doesn’t do anything!

If you’re wondering, the eval comes in the next line which isn’t here.

Principle 17: Code should do what it seems to be doing (especially when it seems to be doing subtle).

Principle 18: Conceptual elegance is no guarantee of actual maintainability (nor a good substitute for it).

Principle 19: If you’re going to have default values, define them near the place they may actually be used (or at least in some place where they’ll be easily found).

Principle 20: No matter how good you think your error messages are, they’re still too brief, too obscure, and too hard to decipher (even if you’ve already taken Principle 20 into account).

Principle 21: Avoid using obsolete and arcane magic punctuation variables with unfamiliar default values and unexpected global effects (even if you happen to enjoy a little self-inflicted pain in other recreational situations).

Principle 22: The fundamental complexity of any problem is irreducible (optimisations merely redistribute the pain differently).

Principle 23: Code that breaks when it’s reformatted is already broken (though on a much more profound and interesting level).

Principle 24: If it’s impossible to understand, it’ll be impossible to maintain (on the bright side, of course, such code is highly stable).

A computer science quine is a program that prints its own source code by concatenating (partial) copies of itself. Similar to a linquistic quine, such as “added to itself yields a sentence”.

Principle 25: Phenomimetic retrodeterministic nominativism does not improve code comprehensibility (then again, did the name make it sound like it might?)

“Far too few people are using their source code as a user interface.”

Principle 26: Don’t allow dynamic behaviour to violate static expectations (and the easiest way to do that is reusing over-scoped variables for unrelated purposes).

One of the most beautiful lines in SelfGOL: $"=",";. $" is the array stringification separator, and is normally a single space.

Principle 27: Explicit behaviours are better than implicit behaviours (especially when the specification of the implicit behaviour is syntactically baroque and hard to spot, and the behaviour itself is unknown to the majority of programmers).

Principle 28: Code that pre-caches or precomputes its data is much easier to maintain than code that caches or computes on the fly (when you’re running at multiple gigahertz, acquiring your data a few thousand operations early is still plenty JIT enough).

The line of Perl Damian is most proud of writing: y=[====y=]==||&d. With whitespace and substituting in “standard” delimiters, this is y/[// == y/]// || &d. Count up the number of matches, and if there aren’t an equal number of opening and closing parentheses, then call the d() function. There is no d() function, so Perl throws an exception, and the eval catches the exception, and prints “No”.

Principle 29: Coding is an art, but code shouldn’t be art (evolution made programmers boring, pedestrian, and aesthetically challenged for good reasons).

Further reading:

• OSCON 2008: Perl Worst Practices via sirhc.us maximus
• OSCON: Perl Worst Practices via The Daily Ack

[tags]oscon, oscon08, perl, damian conway[/tags]

1. but what doesn’t [↩]

Paul Fenwick is from Perl Training Australia.

We start off with one for Tim and Frossie: “Never allow a rabbit in your machine room!”

Most security is common sense. Unfortunately common sense is not that common. There are some times you should be thinking about security: when running a program under extra privileges, when you’re dealing with sensitive information, and when you’re dealing with untrusted data.

There are three general rules: when you can, drop privileges, wipe secrets and never trust external data.

Perl has some extra features and unique issues that other languages don’t have. If you want to write secure code, do it with a friend. Code review is fantastic. Also, use strict and use warnings (and use autodie as well). You should be commenting your code and using descriptive (but terse) variable names. Use CPAN, and stay patched and current.

Note: security is a moving target. Tips today may not necessarily work in the future.

Computers typically deal with GIGO. Security deals with EIEO: Evil In, Evil Out. An example is when you try to move the UNIX finger to the web: you need to make sure that the input “username” to finger isn’t something like “fred; rm -rf /*“. This is called “data verification” — you should always check your input. This means input from anything: users, files, other programs, etc.

Luckily Perl has taint checking, which is turned on by the -T command-line switch. Any data coming from an external source is considered to be tainted, and so is any data that deals with data coming from an external source. Taint mode gets turned on by default when running under setuid or setgid. It also checks to make sure your environment is safe by testing various environment variables, and it ignores the PERL5LIB environment variable.

Fundamentally, it's designed to not trust external data. But our programs typically want to use external data, and if it can't be used, then our programs become useless. Luckily external data can be cleaned. In Perl this is done using regular expressions. Captured variables in regular expressions (i.e. $1) become cleaned. The original variables are still tainted, but the new ones aren't.

There can be problems with untainting data, however. You could screw up your regular expression so later processing goes completely pear-shaped.

Because of this, always check for success. When you run a regex, check to make sure it succeeded:

my ($clean) = m/^([\w.-]+)$/;

And don't get caught in the belief that just because a variable is untainted, that you can trust that data. If you use re 'taint', then regular expressions will no longer untaint your data. Then in the section of your code where you want to clean up data, use no re 'taint' in a block (re 'taint' is lexically scoped so you can do this).

You can accidentally untaint your data. Taint mode does not provide any restrictions against printing out sensitive information. Taint mode does not check anything when you're opening files. Taint does not prevent you from using a variable as a subroutine reference. Taint mode doesn't kick in in multi-argument system or exec calls in Perl 5.6.

Data validation can be hard. Always deny by default. Write tests. You can discover if a variable is data by using the Scalar::Util module. You can discover if taint mode is on if ${^TAINT} is true (in Perl 5.8+).

Perl is good for making simple tasks easy, like opening a file. Unfortunately TIMTOWTSIU (there is more than one way to stuff it up). If you're opening a file, always specify the mode, and always use the three-argument version (i.e. open( my $fh, '<', $filename). But just because you're using the three-argument version, it doesn't mean you're out of the woods (you can still have $filename = '../../../../../../etc/passwd'). Validate your data!

Symbolic links are awesome! They're useful but you can use them to do evil evil things. Most programs chase symlinks, and this can be exploited, especially when dealing with temporary files (because /tmp is world-writable).

Race conditions are also a problem:

if ( ! -e $config ) {
  open( my $fh, '>', $config );
  ...
}

You can sneak something between the if and the open that'll screw things up. Instead, use sysopen in combination with the Fcntl module.

A common attack vector is file names. Perl can use anonymous files, which points to a file on disk that doesn't have a name:

use autodie qw/ open /;
open( my $fh, '>', undef );

...or use the File::Temp module, which is available in core from Perl 5.6.

system() and backticks are horrible. There are many ways you can get them wrong. So many people like Perl because system() is really easy to use. Unfortunately using it right is hard. The single-argument version of system() goes via the shell; the multi-argument version doesn't -- most of the time. system( $cmd, @args ) doesn't go via the shell if @args is empty!

And never never ever ever use backticks. They always go to the shell. There's no way to stop backticks from going to the shell.

Instead of using either system() or backticks, use IPC::System::Simple, which has a command called systemx. capture and capturex replace backticks (and actually die if things fail).

A setuid program runs under the privileges of the owner. An example is the passwd program. A setgid program runs under the privileges of the group.

Don't use perl's setuid(), because it's implemented differently on different systems. Instead, use setresuid(). But to do this you need to do osmething likesyscall(SYS_setresuid(), 1000, 1000, 1000); Unfortunately in Perl $< and $> are cached, so they can lie to you, so this is a non-good way to test for what user you're running under. You could use Proc::UID, though, but it's not implemented for a few systems.

When dealing with DBI and SQL it's easy to open yourself up to a whole world of problems. Don't ever trust data from users. Ever. There's an easy solution to this: placeholders.

You can also set an extra flag when creating your DBI object so that DBI is taint-aware. You can also set TaintIn (cannot send tainted data to DBI) or TaintOut (any data coming from DBI is considered to be tainted).

A few tips and traps:

• The double angle bracket diamond operator (<>) uses the two-argument open(), which we know is unsafe to use. It can be exploited to execute code. It also doesn't get "fixed" by turning on taint mode.
• Perl strings can be any length. They can also contain any character you want -- including Unicode, control characters, and null bytes. Since Perl is written in C, if you can get the null byte down to the C layer, you can potentially cause havoc. An easy attack vector is through URL handling -- the null byte in a URL is %00, which could get through URL verification quite easily. Taint checking helps with this, but it doesn't solve all our problems. Easiest way to fix things: don't accept the null character! Do a default deny instead of default accept on characters.
• Bad untainting: using a tainted variable as a hash key.

CGI::Untaint is a good resource for untainting data -- not just for CGI data.

Further reading:

• OSCON: Perl Security via The Daily Ack
• OSCON 2008: Perl Security via sirhc.us maxim.us

[tags]oscon08, oscon, perl, perl security, security, paul fenwick[/tags]