Back in June I went to Yet Another Perl Conference 10 in Pittsburgh. I took notes at the time but never put them up here. Well, it turns out that I have to give back my work laptop and since I don’t want to lose the notes, here they are in one giant lump.

*** YAPC 10: Opening Address

protip: when going to a different country, take your passport

some story about flying for the first time (turboprop)

*** YAPC 10: Richard Dice, president of the perl foundation

A Year In Perl: Update since last YAPC::Chicago

tpf's mission is to support the perl technology and community
tricky to know how to do that sometimes, lots of different voices in the community (and in tpf)
tpf does events (yapc::na, hackathons, workshops)
tpf does grants, community legal work (managing IP, handling lawsuits)

finances and donations:
 - hague donation: $200k (100k for perl6, 100k for tpf internals)
 - general donations: $70k (ad optimers llc 25k, anonymous 25k, all others 20k)
 - booking.com: $50k

 grants:
  - 5.10.1 release grant $5k to david mitchell
  - 5.8.9 grant help to Nick Clark
  - hague grants for perl 6 implementation efforts
   - increase # of p6 implementation devs
   - increase productivity of devs
   - increase amount of time that devs can dedicate to p6 imp. efforts
  - daniel ruoso, SMOP, hague grant
  - patrick michaud, rakudo perl development, hague grant
  - jerry gay, s19 command line syntax, hague grant
  - jonathan worthington, traits (s12 & s14)

 parrot was spun off from tpf in march 2009

 vienna.pm hosted yapc::eu::2007, surplus frm that (about 25k euro) goes to jonathan and patrick

 tpf is all-volunteer, time-constrained
 hague grant goes towards tpf work (possibly full-time work for richard) details in next few days

 grants committee reviews and ranks grant proposals on a quarterly cycle
 5 grants in 2008q2 q3, 2 in q4 2009q1, 1 in 2009q2

 google summer of code
  - annual grants to university students
  - nine projects this year (five last year)

 perl at large
  - perl5 uses git for version control
  - strawberry perl for windows
  - "perl iron man"
  - japan perl associatin
  - enlightened perl organisation
  - modern perl books, a modern perl blog
  - moose
  - padre ide
  - november wiki

 perl events
  - oscon july 2008
  - pittsburgh perl workshop oct 2008
  - perl oasis orlando january 2009
  - frozen perl workshop minneapolis feb 2009
  - yapc:na june 2009
  - six in CIS, 3 in asia, one in australia, one in brazil
  - 11 workshops & hackathons in europe

*** YAPC 10: Larry Wall

Larry liked old error messages, like UNIBUS TIMEOUT FATAL TRAP

would you get on an airplane if you knew the avionics were written in perl?

paranoia is necessary for modern life.

perl5 is very powerful and very extensible. perl6 is very very powerful and very very extensible. do not trust perl6.
perl6 is very chatty for syntax errors

Larry just went through error messages in perl6

*** YAPC 10: Jerry Gay, The Parrot Foundation

Why should I care about Parrot? many languages can be run on the parrot vm

*** YAPC 10: Yaakov

What is YAPC? Go meet people

** YAPC 10: Michael Schwern - Trapped In A Room With Schwern

schwern isn't a linear thinker, so this talk might meander

what do we want schwern to talk about? two-part question: what are you working on next year? what do you not want to work on next year?

second answer first: PHP first answer second: he wants to be the redhat of cpan

wants to take the 5000 modules and 2000 programmers and have one front-end. you want things to just work, you don't know what you're going to get tomorrow. love to do this as a commercial package

he's very tempted to declare make::maker done, go use module::build

good things about php: the thing php does right is you can take it out of the box and it works. don't need to install modules, compile things in, etc., it's all there. it has an OO system, not perfect, but it works. 

bad things about php: 75 built-in functions to deal with arrays. 13 different sort functions. 

perl 5+i. your perl ends up with a whole bunch of 'use module' lines at the top of your program. this is "tribal knowledge", where you have to go and hunt to find it. you'd rather do "use perl5i". this exists on CPAN now, but you can do things like:

  use perl5i;

  say "Hello";
  say gmtime->year;
  say $hash->perl;
  open my $fh, "<", "asdfadsf"; # autodie
  $hash->keys->sort->join("\n")->say; # autoboxing

These become methods on a scalar object, array object, etc. thus they don't pollute the global namespace. http://github.com/schwern/perl5i

agile dating: schwern is polyamorous. like poly, nobody's doing agile programming right. agile == polyamorous. when you say you're poly or agile, it doesn't really mean anything. 

you don't know what you don't know, so it's important to go out and discover new things.

*** YAPC 10: David Fetter - You can do THAT without Perl?!?

looks like this talk is SQL-oriented.

lists: windowing functions. let you access the set of rows from the current row. new windowing functions. 

I thought this was a Perl conference.

Oh man I completely messed up, I thought he title was "you can do that with perl" and not "without". Whoops.

there is some interesting stuff here (recursion in SQL)

jesus christ he just made SQL create a mandelbrot set fractal graphic

fetter.org will have the talk (so will pgexperts.com)

*** YAPC 10: second half-ish of "Git Is Easy" by Ricardo Signes

how much to commit at once? git is really really fast, so you feel free to commit a lot more. this helps make each commit "one thing". you can squash commits together, if you want to. You can rebase interactively which can split or combine commits (oh wow)

bisect to find bugs. someone reports a regression (something worked before and it doesn't now). you don't really know when it started happening, but you have a failing test. git can find the bad commit for you by using bisect.

 git bisect start
 git bisect bad
 git bisect good 1.234

*** YAPC 10: Mike Schilli - Moving CPAN module projects to github.com

github is not a cpan replacement, it's more like sourceforge. there's a low barrier to contribute.

create a new repository. give them your public key. boom, a new project.

how do you name your project? libwww-perl, log4perl, p5-Net-Amazon, perl-ldap are all examples of names of projects on github. no naming convention, so go with whatever you want.

every time someone wants to contribute to your project, they click the 'fork' button. kind of like branching. the other person then gets write access to his forked version. he can then send you a pull request to get his patch into your master repository.

when you tag on github it creates a tarball, which you can tell PAUSE about and upload to make a new release

*** YAPC 10: Barbie - Stats of CPAN

7477 CPAN authors registered with PAUSE
4460 who've uploaded distributions to CPAN
~450 authors uploaded to cpan last month, up from ~125 in jan 2000
~40 authors uploaded their first distribution last month (stayed steady from ~jan 2001)
18085 distributions on cpan
55409 distro versions
20304 distros on cpan and backpan
112037 distro version on cpan and backpan
4054659 testers reports
16 perl versions and 57 platforms tested during may 2009

further reading: CPAN testers statistics and CPAN leaderboard websites

*** YAPC 10: Hans Dieter Pearcey - Dist::Zilla - Automating Quality Since 2008

everything you use has the installer and distribution packager being the same thing (make install and make dist). what happens if we untangle them? use Dist::Zilla, you don't need to put all the boilerplate POD in, just create a dist.ini that has the relevant information in it and it'll add all that when you make your distribution

*** YAPC 10: CLI apps don't have to suck, by Christopher Nehren

lots of wheel reinvention in CLI apps (tons of modules related to parsing command-line options, for example)

getopt modules are (for the most part) fairly simple, but like any system, software tends towards compleity. this means that simple tools don't do the job anymore, and things don't scale well sometimes. (woah, he used git as an example of something that has a good command-line interface)

cli apps are critical, so make them testable. functionality can get reused, so make them modular. 

App::Cmd

demo of code which I'm not going to write down

there's also App::Cmd::Simple

*** YAPC 10: Take Advantage of Modern Perl, by chromatic

if you're happy with perl as it was in 1999, this tlak is not for you. ditto if you want to see code.

modern perl is a name chromatic has put on a concept for perl circa 2009. perl is a good language, has a great community, cpan is a shining gem. 

drawbacks of perl:
 - perl5 is seen as an upgrade path for perl6
 - we have a huge legacy code problem, not just legacy code, but truly awful legacy code
 - external support is pretty awful
 - there's a lot of uncertainty in the perl development process

the modern perl plan: want to ameliorate the problem of legacy code. want to replace bad examples and tutorials with good ones. promote sustainable development practices. advocate an improved language and ecosystem. want us to make many different mistakes with perl6 (understand and do not duplicate the mistakes in perl5)

there are a lot of other people thinking the same thing: perl initiative, tpf, strawberry & chocolate perl

governing principle of modernity: identify what's painful in a language and ecosystem. ask ourselves how to make the pain go away. the best way to provide modern perl is to provide saner, less prickly defaults

how to advocate modern perl: experiment with new techniques and ideas, help the community to converge on good defaults that make sense, "I don't know how to misuse this, I only know how to use it right". regularly refactor the code you maintain

how to develop modern perl: document your experiments and experiences, reuse and extract foundational techniques, identify common ground that we can take and reuse. 

how to experiment with modern perl: just go use it, you have chromatic's permission

new features you need: modern perl is 5.10 and 5.10.1: given/when are his favourite features. use the switch statement, not the Switch module. say is the most useful addition. defined-or is nice too. named captures are "very very nice" -- you avoid the gropuing parentheses problem and they make things more readable.

some extrenal modules you should consider using: SUPER, UNIVERSAL::ref (ref() is broken), perl5 is not y2038 compliant so use Time::y2038, CLASS, autobox (not everything is an object, but autobox can make things objects automatically: 'hello world'->ucfirst->say), Moose, Method::Signatures, Exception::Class, Modern::Perl (chromatic believes it's provocative), use roles with Mmoose (google the-whi-of-perl-roles on modernperlbooks.com), Module::Build (ExtUtils::MakeMaker is a horrifying mess its maintainer put on life support /years/ ago, but it will not die), autodie (will be in core in 5.10.1), Devel::Declare, MooseX::Declare (if you use Moose), Devel::NYTProf, Perl::Critic, 

what's missing in modern perl? no good way to say "I want to install Catalyst". no good way to do GUI programming. a non-lovecraftian xs replacement (no-one in this world should write XS). make taint less all-or-nothing.

towards a modern future: perl5i, modern perl: the book

*** YAPC 10: To Moose or Not To Moose, by Stevan Little

reasons why and why not to use moose,where to use it and where not to use it

moose has been around for about 3 1/3 years, about 80 releases, about 2-3 patches per month (fast moving, they want you to keep up-to-date as much as possible, fast bug fixes), 60+ contributors

moose depends (directly) on 8 modules (devel::global::destruction(?) amongst them) which depend on about 21 modules. 488 CPAN modules depend on Moose, 157 different module authors use Moose on CPAN, ~180 users on #moose, over 6000 unit tests (which means you don't have to do things like test your accessor methods because Moose already does these tests itself)

the bad stuff: 0.24 seconds and ~3mb of ram do do 'perl -Moose' (that's the overheaad), basic moose use creates lots of objects (at least seven), don't use it if you have hard time requirements, don't use Mouse.

*** YAPC 10: perl5i: Perl 5 Improved, my Michael Schwern

perl5i comes out of what chromatic talked about in modern perl. whenenver you look at someone's code (someone new, not part of the community) "you should be using this feature, that feature, etc etc" and the etc etc gets really long. it's tribal knowledge -- the community knows it, but the new person doesn't, even if they read tutorials and books.

the first goal of perl5i is to take that tribal knowledge and slap it into one line: use perl5i;

"if you have to keep looking it up, it's a bad interface" -- see localtime() and struct()

turns on strict and warnings along with 5.10 features

turns time things (gmtime, localtime) so they return DateTime problems

fixes the y2038 problem

brings in Alias

turns on autodie

brings in Data::Dumper

turns on autobox

*** YAPC 10: Driving a USB Rocket Launcher from Perl in User Space‎, by Michael Schilli

USB is a pretty versatile technology (panic button, memory sticks, rocket launcher)

drivers usually only come with a windows driver, which doesn't get you very far in Linux

the kernel has a 'usb core' thing that simplifies the hardware interface. there's a thread in the kernel that does nothing but pay attention to devices being plugged and unplugged, which can notify aplications in user space

device drivers are tough to write and are highly dependent on kernel version. vendors don't want to bother with niche markets, which is why they're usually only available for windows (and sometimes macs). usbfs puts a bunch of things into the filesystem.

libusb talks to usbfs, which indirectly through the usb core talks to the usb hub

Device::USB can be used to talk to your USB device from perl.

you probably need to reverse engineer the windows driver (protocol sniffed with usbsniff, dll investigated with ida pro disassembler). control transfers for movement and firing, bulk transfers for reading status

device::usb::missilelauncher::rocketbaby

*** YAPC 10: What Haskell did to my brain, by  Yuval Kogman

*** YAPC 10: Catching a ::Std - Standardisation and best practices in the perl community‎, by Matt Trout

what is a standard? it's a word that gets used for so many things but loses its meaning (like modern or enlightened).

there's a sort of standard for pronunciation (bbc english) OED standard reference for the english language, but there's also Webster supplying another standard. dialects are interesting because you use different grammars but the same semantics. 

standards in perl? "whatever the interpreter does" is the standard. how do we check that? "whatever the tests say"

standards for writing perl? TMTOWTDI. problem

there are a number of different approaches you can take: inferred standards, invented, enforced, evolved.

inferred: UUCP is inferred. perl is effectively an inferred standard. we know basically how it's going to behave, but that's be observing what's happened in the past.

invented: TCP/IP (intelligent) IMAP (possibly one of the ugliest to implement)

enforced: some specific body has control. java JCP, SMB (whatever the most recent of windows does)

evolved: POSIX, Scheme

standards and perl6: he's a skeptic of the perl6 production implementation but is a fan of the perl6 language design process. it's enforced by the standard grammar, and it has evolved based on implmentation and use attempts. 

"best practices" means even less than "standard". "bunch of things some java guys used on some projects and didn't get fired" damian tried to invent a standard, and a lot of it was good, but Class::Std isn't apparently.

why did class::std fail? it had a bunch of good ideas, but damian ran out of tuits. but... it tried to be a revolution, but it was incompatible with the stuff that was already out there. needed everybody to move and reimplement their existing stuff to make it happen. it's also kind of unperlish.

cgi::application could have been a standard but didn't quite make it. it's kind of too little of a standard. plugins are a free-for-all and ends up being a mess. not cohesive, two different cgi apps feel completely different

maypole had a sane api but overly tight coupling with other things it thought should be standards, but class::dbi imploded and took maypole down with it (mostly)

dbix::class is kind of the common ORM now, wasn't meant to be but then people deployed from trunk, so he had to make stable releases. 

the best way to get a standard is to only call it the standard when enough people have spread it via word of mouth

enlightened perl extended core is an interesting example of people trying to do that. it's evolved from "what do the experts consider their standard toolkit?" is it the One True Way? not so much, but it's One Good Way.

*** YAPC 10: Oops! I i18n'ed Your App, by Jeffrey Goff

*** YAPC 10: Perl in the Time of Social Networks, by David Moreno

"I used to be a debian fanboy" some guy in crowd goes "yay" "...but now I use a mac because it lets me get things done quicker" haha

social networks are complex websites and online communities that bring people with common interests together to bond and connect. people love connecting. they enjoy spending time playing (mafia wars  "addictive piece of crap" ) why are they importnat to people? they are the websites that bring the human side to the internet. 

three major social networks websites: facebook, twitter, and myspace (at least in the US)

facebook: it has generated $300m in revenue, started with $500k funding. 307m users worldwide

myspace: horrible website. was worth $580m when news corporation bought it. got overtaken by facebook in april 2008. 128m users worlwide in june 2009. is myspace dying? it's lost users, but there's still a huge userbase. there's also myspace music which allows indie bands to promote themselves easily. google paid $900m to provide search.

twitter: biggest assets is the media awareness. because of that and because of celebrities (obama, ashton kutcher, oprah, shaq). nielsen ranked it as fastest-growing social network: 1382% monthly growth (that doesn't sound right). it has persistent outages, but that hasn't hurt them that much. us airways flight 1549. #iranelection.

why should you care? lots of business opportunities. easy to reach a lot of people. 

the three big ones have APIs, so use them. they're all RESTful resources. facebook has FBML and FQL. myspace has implemented most of the opensocial stuff. two different myspace dev sites: myspace apps and myspaceid (for external apps). twitter has implemented oauth.

there are a bunch of frameworks in perl, like http::engine and catalyst (or modperl if you have the patience)

there's www::myspace, but it lacks a lot because it's mostly webscraping. has nothing to do with the opensocial specification, but there's net::opensocial::client::container::myspace, but there will soon be net::myspace. 

*** YAPC 10: Perl in Vegas, by Scott Walters

all the security was put into place to crack down on organized crime that came to power during prohibition. 

** YAPC 10: Meet Melody, by Timothy Appnel

melody is a fork of movable type open source. it's available on github.org/openmelody.

there are other blogging software packages and tool that get more press, but the number of large and influential blogs running MT is quite high (the overwhelming majority) tpm, boing boing, huffington post, scienceblogs... a number of corps use it as well: intel, boeing, amex, gm. mt's dominated that realm. perl pride.

it's been around for 8 years and there's lots of good stuff in it. uses a static publishing model. very low overhead. six apart released core of mt under gpl. with melody they're taking that a step further.

what is that step further? what's going to be different? right now, not much. what they're trying to do comes down to one word: community. a community didn't rally around mt as an open source application, but now htey're trying to change that. 

501c3 being formed to protect the IP via gpl, hold contributions in a trust, support the greaty melody and mt communities.

*** YAPC 10: Perl5 Memory Manglement, by Steven Lembark

why you care: long lived, high-volume, or high-speed apps need to avoid swapping, etc

standard intro to perl's guts is perlguts, also includes perldebuguts to use debugger

perl is simple: always favours speed. as the programmer, you have to manage the space

perl's memory model is similar to C: subs have a scratchpad which is like C's stack, and a heap for permanent allocations. lexicals live in the pad but are allocated on the heap. perl's heap is expanded to meet new allocation requests, and it does not shrink

most people think of "size" as data:
 length $string;
 scalar @array;

this leaves out the structure of the variable itself, all of the wrapper stuff around the variable. devel::size uses size for the structure, total_size for everything

strings are allocated as c-style strings. they grow by being extended or re-allocated. removing characters from a string either moves the 'start of string' pointer or reduces the length counter (or both). structure doesn't shrink.

the way to get your memory back is to assign it to a new SV. mangle strings and then assign them to a new variable. there's no other way to do it other than to use lexical variables to return memory.

takeaway: strings don't shrink

arrays are similar to text. they're reduced by adding a skip count to the array and reducing the length. leads to confusino if you try to 'freee up space' by shifting data off an array

shift and pop don't do anything to the amount of memory allocated. in fact, if you 'shift' then 'pop', the resulting array is larger than the original

perl actually reallocates memory to an array in leaps -- it'll give more than necessary to avoid having to allocate each tiime you might need it, because of speed considerations

hashes are bigger than arrays. deleting keys does not reduce the structure, neither does %a=(). collision chains are arrays, so they don't shrink.

*** YAPC 10: Getting Started with Multithreaded Perl, by Walt Mankowski

why should we be interested in threads? because of moore's law. can't really buy a cpu faster than about 3ghz now, so they put more and more cpus on one chip instead. but to run faster you need to be able to use more than one cpu at a time.

the way we do this is with parallel processing: pthreads, MPI, OpenMP, Linda, Java threads, shell pipeline, fork, semaphors, shared memory, pipes, sockets, etc...

threads vs processes: in general (but not necessarily in perl) threads are lighter-weight than processes. faster to create, faster to switch. easier to share data between threads, shared data is in same memory locatin (C model). harder to debug threads: race conditions, locks, randomness

history: Threads.pm (5.005) they really sucked. perl itself wasn't designed to be re-entrant, had to make everything internally thread-safe. removed in 5.10. threads.pm (5.8) introduced in 5.8, improved in 5.10, "interpreter threads" (ithreads) one Perl interpreter for each thread.

creating threads: 

 use threads;
 my $t1 = threads->new(\&tsub);
 my $t2 = threads->new(\&tsub);

 $t1->join;
 $t2->join;

 sub tsub { print "I like pie\n"; }

you can have different return contexts from threads

how to tell if threads are enabled: they're not compiled into perl be default, but many distros build Perl with threads. to check: 'perl -V' and look for 'useithreads=define'. programatically you can do 'use Config; if( $Config{useithreads} ) {...}'

sharing data between threads: all variables are private by default. variables must be explicitly shared, which makes thread creation relatively expensive.

 use threads;
 use threads::shared;

 my $foo :shared; # compile time
 my $bar;
 share($bar);     # run time

scalars arrays and hashes can be shared. a shared variable can't store a reference to an unshared variable:

 my ( $foo, $bar, $baz );
 share( $foo );
 share( $bar );
 $foo = 42;     # ok
 $foo = "pony"; # ok
 $bar = $foo;   # ok
 $foo = $baz;   # ok
 $foo = \$bar;  # ok
 $foo = \$baz;  # not ok

locking: lock as minimally as possible. lock() releases the lock when the current block ends. locks are limited to scalars though...

semaphores: Thread::Semaphor. you can use these for dealing with more complex data structures.

perl threads suck: memory usage is sucky, many xs modules aren't thread-safe, core isn't designed to be thread-safe. could look into forks.pm: same syntax but uses fork() instead.

more info: perlthrtut, threads, threads::shared, Thread::Queue, Thread::Semaphor. o'reilly has Pthreads Programming. computing .llnl.gov/tutorials/pthreads openmp mpi. iTunes U, Stanford Computer Systems Colloquiums, David Patterson, "Computer Hardware Is Back"

*** YAPC 10: perl, cloud glue? by Robin Darby

(started six minutes late)

he's with amazon web services

CloudWatch: monitoring services for ec2 AMIs, loadbalancer, and autoscalinggroups. demo with really small text

Elastic LoadBalancing: oh man another demo with small text. 

sounds lke these examples are going to be on the web

http://developer.amazonwebservices.com/

http://aws.amazon.com/documentation/

AMI-27e2004e

*** YAPC 10: ‎Object oriented perl -- everything you were too embarrassed to ask, by Leonard Miller

really objects for people who haven't used objects in perl before. if you don't know what $self means. data::dumper is your friend when trying to figure out what objects are doing.

$x is the same as ${x}. @a is the same as @{a}, so you can do @{[ fn(1), fn(2), fn(3)]} and:

  print "the function returns @{[ fn(1) ]}\n";

*** YAPC 10: Data Visualization with Chart::Clicker, by Cory Watson

slides on twitter.com/gphat

make simple things simple, make hard things possible, make it all pretty, and make it possible to follow Tufte's thoughts.

*** YAPC 10: Enlightened Perl

Why is it called Enlightened Perl? enlightenment is a period of rapid change on thinking. perl has evolved and developed more as a language over time. mature CPAN, mature programmers (reusable, stable, tried to support communities) mature communities

there's nothing wrong with timtowtdi, but consistency is nice sometimes

what makes an EPO project? it's an extension of perl itself, extension of perl OO. stable, production ready. lots of testing. documentation. keeping in touch (mailing lists, irc, forums, etc). use version control.

what EPO projects are not: a toy or hobby