I work as a professional Perl developer. We have a large and complex system which is built around a lot of Perl. This system has been in use and in place for years, and handles an astonishing amount of load, using thousands of computers to do it.
I work with some excellent engineers, who are thoughtful, intelligent, hard-working folks. They want to make the best system they can, and they want it to be reliable and dependable. I have great co-workers.
I keep finding Perl that looks like it was written in 1990. I find code like this:
sub logger {my $msg = shift;open(FH, ">>$runpath/log/output.log ") || die "Can't write output.log!"print FH "$0: $msg\n";close(FH);}
(Edit: Fixed the mistake that made the sample useless. Mark Dominus pointed out in the comments that you can’t write to a read-only file handle. << was wrong.)
The first dozen times I found code like this I thought to myself, “Ugh, old Perl. Been here for ever.” and cleaned it up. This is an old codebase – parts of it still have to be compatible with Perl 5.6, for instance – and these constructs weren’t so badly dated when the project began.
I had to stop thinking this while doing a code review of a new change. Filehandles used with the two-parameter open in brand-new code.
I stopped and asked the engineer why they’d done this. They looked at me like I was insane. Of course they’d done it that way – that’s how you open files in Perl. I had questions: “What about using a lexical instead of the typeglob?” “What about using the safer, clearer three-parameter form of open?”
I got blank stares. None of them had ever heard of these changes to open.
I’ve been educating them, which has been interesting. I’ve also been thinking about how we got here.
There’s two things I see that led to this situation, and both of them are bad for Perl.
First, many of these engineers aren’t aware that Perl is still being developed. Perl 5.8 was so stable for so long, they’ve quit watching. They think it’s done, and there’s nothing new under the sun. They bought “Learning Perl” and “Programming Perl” in 1991 and are happy with both of them. Neither of those books mentions lexical filehandles I realized the copy of “Programming Perl” on my own bookshelf didn’t.
Second, it seems like lots of people quit learning about what they’re using. It’s done, carved in stone, and that’s how they use it. The idea that the Perl community exists is odd to them, and apparently irrelevent. The ideas that Perl’s idioms might change or that new libraries, tools, and language constructs come along… these are hard ideas to convey, apparently. The question I keep getting asked it, “What’s the benefit for making these changes? What I do works, why should we change?”
Luckily for me, the project leads and our managers do understand why keeping up with the language is important, and I’m not swimming against the current to try and be bringing this new information to the team. Even so, there’s a couple of big rocks we’re having to go around.
How can the Perl community get past this huge piece of old information about the language? There are a million web sites with old examples, and thousands of outdated copies of the O’Reilly books on bookshelves. What can the community do to make it widely known that these are old, and that Perl Is Not Dead and that not only is There More Than One Way To Do It, there’s probably a Better Way now.
I worry that the addition of Perl 6 to the mix will actually make this worse. Searching for it will be hard, and how will you find examples for one or the other?
I honestly wonder if *both* languages shouldn’t change their names. Leave Perl 5 in maintenance mode forever and branch it to a new language, even if it’s just Perl 5 with better documentation. Take that time to reconsider the core modules, clean up some historical messes and release it as something new. Something that can still use CPAN, can still run all your Perl, but has active development and real, usable tools. I’d give Perl 6 a new name, too, so you can search for it properly.
Or maybe that’s the worst idea ever. I don’t know. I worry that Perl is in a bad spot now, and is trapped in the past.
Tags: Perl
How about sharing the brand new book “Modern Perl” with them: http://www.onyxneon.com/books/modern_perl/index.html?
I will be doing so. I didn’t know it was finished yet! As soon as I saw your comment here, I ordered a paper copy, downloaded the PDF, and sent chromatic a few bucks. So far, it’s an excellent book.
Are you kidding me? Excellent? Right now i am on page 40 and it’s full of errors, shitty code and bad examples
Then you’ll certainly not agree with my longer review at http://blog.laufeyjarson.com/2010/11/shiny-new-perl-modern-perl/
I invite you to point out the things you’re finding wrong with it there, so we can discuss them in detail. I certainly didn’t find the same things in the book.
“Learning Perl” has edition that covers lexical filehandles – 5th. “Modern Perl” book was released this month. I’d also suggest giving then perlcritic tool.
My point about “Learning Perl and “Programming Perl is that these developers already have copies. I discovered I was doing the same thing, that the copy of Programming Perl I use is really outdated.
I’d been keeping an eye on the Modern Perl book – I didn’t realize it had finally shipped. I’ll definitely be getting a copy and passing that knowledge around the office.
Getting people to realize the tools are changing has been harder than expected.
In our team all of the developers are aware of the Perl community and the fact that Perl is still under heavy development, even if we’re still stuck on 5.8 (for the moment). The reason for that is that two of our four senior developers (and our development manager, who used to be one of those senior devs) are heavily involved in the local Perl community and keep up with what’s going on. And from there we have two purposes: acting as gatekeepers to make sure no “outdated” code enters production, and to make sure that everyone knows whats new and important for them.
Because we’ve done this for years we have established a bit of a culture around the place, and our developers at least grudgingly accept that they have to learn and update or their code won’t make it in, though most are happy for the input and guidance. It sounds like you’re in a similar position in your organisation, so perhaps you could position yourself similarly? In other words, you can be the Perl community for your team.
“Professional development” is a problem in every field. If you don’t have an attitude and culture of constant education, there’s nothing that anyone can do to help. That is, there’s nothing that the Perl community can do for people who aren’t paying attention in any way.
Perl certainly isn’t trapped in the past. The people who are using it might be, but that’s not the same thing.
I have recipe that can work on few co-workers: use natural evolution. It can require some impulses, eg. Christmas coming so buy _few_ books and drop them on shelves in your company. Allow co-workers use that boxes as Rubic cube – use a little, throw one to each other, burn if they want, etc. It’s not magic, just process of knowledge assimilation.
“Modern Perl” was sugested. Second can be “Refactoring to Patterns” by J. Kerievsky – hovewer examples are in Java but book influence on thinking about old code can be great.
Those developers, not Perl, are trapped in the past. There is little anyone else can do about it, but to point them in the right direction.
The Modern Perl book is a good advanced introduction, also http://perl-begin.org/tutorials/bad-elements/ attempts to point out to several common mistakes.
Been nice if you included the changes in the article for us old perl programmers who also wanted to see what you are talking about.
That is an excellent point. I fear that formatting in the comment will make this ugly, but here goes:
sub logger {
my $msg = shift;
open(my $fh, ‘>>’, “$runpath/log/output.log “)
or die “Can’t write output.log!”
print $fh “$0: $msg\n”;
close($fh);
}
(EDIT: Fixed the mistake, as Mark Dominus pointed out in a comment. Wrong file mode. My error.)
There are two major changes to the sample from the text.
One is that instead of using a filehandle, you can use an ordinary lexical. These are easier to work with, can be passed to functions, can have references taken, etc.
The other is that there’s a form of open() with three parameters, moving the type of open you’re requesting out of the filename and into a separate parameter. This helps prevent nastiness where someone provides “|/bin/sh” as a filename. It can often save an interpolation, although this example doesn’t.
I also switched from || to ‘or’, but that’s more style and preference than old vs. new Perl.
You really should be using autodie. This checks the result of all system calls for you. In the code above, you should be checking the return value of the print and close, not just the open. But again, just use autodie.
The ‘new’ way is to use one of the logger modules (I like Log::Log4perl) instead of rolling your own logger.
I still use the “old” style of opening file handles. Didn’t even know there was a different way of doing it. I write odd quick scripts to automate stuff, when something like this works:
open (my $file,”>destination.file”);
why would you expect me (or anyone else) to know or do any different?
As far as my experience goes, most programmers don’t involve themselves in the community: They see the programming language as a tool, not a way of life or social circle. When version changes come they’ll brush up on the changes, take advantage of new features, but in between times rarely bother.
Well, if you were a serious Perl developer I would *expect* you to keep up with current best practices. Otherwise, I would just educate developers as I went.
That’s a strong way to state it,but I was thinking a similar thing.
I don’t keep up with Perl to invest myself in the community and make friends, or treat it as a social circle. I keep up with what’s going on to make sure that I remain as skilled as possible with the tool. It’s an investment in myself and my own knowledge, not a way for me to meet people and do social things.
I started learning Perl in June for a new job, and have seen Perl still being written like your example.
It probably wouldn’t hurt to introduce your colleagues to Damien Conway’s “Perl Best Practices”. I soon realised that my (now ex-)employer adhered to none of these practices. Also check out the Perl::Critic web site http://perlcritic.org/, with the settings at “brutal”.
Also, “Effective Perl Programming”, 2nd ed., by Hall, McAdams & foy.
Why not start using Perl::Critic with a test that ensures that no new violations are added? Perl::Critic can tell the programmers exactly what they are doing wrong and how to fix it.
This very proficient old Perl programmer is wondering what “lexical” means in this context. I’ve seen Larry use the term, but I haven’t a clue what it means. Still my Perl scripts run just fine and are perfectly readable, thank you.
So is this “modern” Perl book going to be kind of like that “Perl Best Practices” book that we are not supposed to use, outdated in six months?
Mark, the Modern Perl book has an excellent description of what a lexical is. I suggest you read it. In short, it’s a “my” variable, but the book does a better job of explaining what the details are and why it is called that.
I understand your frustration with “Perl Best Practices”. I really agree with Mr. Conway’s premise that examining what you do and doing things for a reason is important. I then disagree with many of the conclusions he reached. The book is still highly regarded where I work, and only a few parts are worth “not supposed to use”.
Even in 1990 I don’t think you could write to a file that was only opened for reading.
You’re correct, of course.
This is what I get for typing code into a blog post from memory, without testing it.
Now I feel foolish. I’ll fix it above. Thanks for pointing it out.
You are a humble person. I like you.
I work in Perl on a daily basis and still use the old style to open file handles. I didn’t know there was another way. And yes, I’m running 5.8 on my production machine.
I’m also learning and beginning to work with Python. Yes, I would upgrade my knowledge of Perl, but Perl 6’s 10-year production cycle is a deal-breaker. 5.10+ Perl is an alternative track, but I’m afraid of what it would break. Perl 5 modules are often static and no longer maintained; the developers have moved on to Perl 6 or Python or wherever.
And I think there’s the core of the problem: people who use Perl have lost trust in the people who develop Perl. Is it going to be worth the effort to upgrade to 5.12 or whatever now? Or will Perl 5 simply disappear once Perl 6 is ‘finished’?
We use 5.8.8 in production where I work, as well. The “new” way I suggest has been available since then, and works fine. It should work as far back as 5.6, at least according to 5.6’s perldoc.
Perl 5.10 and 5.12 have been extremely cautions about not breaking existing code. I’ve had almost no problem moving forwards in the Perl 5.x series. There’s been a couple of new warnings, which were easily fixed. There’s also some handy new features – say, given/when, // and //= – which I’d like to use more regularly.
I agree that Perl 6 has caused lots of confusion about Perl. I think it’s been bad for the language as a whole. We’ll see how well it recovers.
There was the idea that Perl 6 was going to replace Perl 5. I think they’ve gotten over that and realized that they’re both valid tools and will both will continue. I don’t see Perl 5 going anywhere soon, as there’s too much existing code and too many good resources in CPAN to give up.
My advice is to ignore Perl 6 until it’s really usable. That includes having a CPAN-like repository full of goodness. It’ll be a while for the libraries to catch up, even if Perl 6 releases this year.
Perl 5 and Perl 6 are different languages. Perl 5 won’t go anywhere.
Python 3 was also developed 10 years (started before Perl 6 and was easier task to do). As for CPAN modules, yes developers move to something else, but new developers continue their work. Static modules can be modules that no longer require any work, like CGI::FastTemplate (popular module fore some time, ported to several languages, at least 3 times to PHP).
I wonder whether the problem is not partially due to the fact that “Programming Perl” has not been revised in 10 years! Sure, there are newer Perl books, but “Programming Perl” has the brand recognition, so if a new edition came out, that would grab the attention of the old timers.
At least that’s my excuse for the lexical file handles. 3 argument open was introduced in perl 5.6, so it should be covered in 3rd edition Programming Perl.
All this whining about not using a lexical and the 3-arg version of open()?
It’s not the end of the world, you know.
A bigger concern should be why there are 15 different ways and modules to do simple stuff.
You are barking up the wrong tree. The only thing really “wrong” with the code you cite is that it neglects to set the encoding. And you haven’t fixed that, either. So your code is just as broken.
If it’s part of a larger program, I’m sure it isn’t going to have a conflict with FH. But it’s pretty dumb to keep opening it and closing it. I’d use a filehandle that wouldn’t run risk of conflict if I were putting this in a module.
Perl::Critic is utter crap, something that should be taken out and shot. These overrighteous Holy::Roller busybodies don’t know what they’re talking about, but they’re sure someone somewhere that conflicts with their Divine Dogma and they are bent on cleaning the world of their perception of evil.
Perl doesn’t need that. Nobody does.
I’ve just been reading “Modern Perl”, thanks to other comments here. It had an excellent example of how this can go wrong, and made it clear that simple handling of data like this is going to bite the programmer.
I did know the encodings were there, but hadn’t realized the criticality of using them properly. I now know they’re important, but have no idea how to do it right. I’m still trying to explain “binmode” to some of the Unix-centric at the office, so may find this an uphill battle.
As to Perl::Critic, I have had luck turning everything off and adding specific checks for things we’ve really had problems with. I agree that the default settings are not too useful.
There’s “wrong” as in buggy and there’s “poor habit in larger programs”. There’s little chance of globals causing trouble in a program of a hundred lines written by one programmer and never maintained after that.
A disciplined or fortunate programmer can avoid those problems, but I hate to rely on novices picking up that discipline in an ad hoc way. Better to borrow the aikido training mentality of shu ha ri, in my opinion.
Tom C,
I also really want to understand why that is important. We all code using Perl with the default behaviors all the time. Why is it important to set the encoding? I looked it up and found out how to specify an encoding, but not which one and why.
Is the below best practice?
open my $fh, ‘>>:encoding(utf8)’, “$runpath/log/output.log”
or die …
Yours,
Tom D.
Tom D,
Chromatic’s book “Modern Perl” has an excellent example of why setting the encoding is so important. (He has very graciously made it available as a download, too, so go fetch it from http://www.onyxneon.com/books/modern_perl/index.html and you can read the section.)
Look at the “Character Encodings” section on page 18. It gives clear examples of how it is easy to accidentally form strings that are not usable in any single character set.
What I haven’t found yet – and I suspect I haven’t found it because it’s hard – is how to get this right and know you’ll be reading and converting things properly. Page 19 of “Modern Perl” suggests perldoc perluniintro for more details, but I haven’t followed that up yet.
Laufeyjarson,
Thanks for pointing that out. I had looked in the wrong portion of the book. I suspect that the easy way to get this right when you have complete control of the data is to always specify the encoding and to always use the same one. I think that means you specify the encoding using a use statement at the top of the file and whenever you open a file.
The difficult part is when you don’t have control of the data. That means you have to find out what encoding is required by the programs / data sources you are interfacing with. Then you can pick the encoding they use (if it is all the same – good luck). If not you can pick the one most use or one which is very versitile and very carefully convert between them.
This may require test cases using unusual characters.
IIRC, Perl 6 is designed to carry the encoding with the string. This helps alot, but you still have to have hope all your Perl 6 code uses encodings which fully translate one to another, and you still have to know the encodings of the non-Perl 6 data sources you interface with.
chromatic and Tom C,
I would love to hear your thoughts on how to get it right.
Yours,
Tom D.
Upon reading perluniintro, I am now thinking my suspicions were wrong. The ‘use open’ pragma looks like a cureall if all your data is encoded the same way.
Oofff.
Yours,
Tom D.
Tom, the more I think about this, the more I think I need to say something.
The use of FH is a problem because it is not a lexical and can cause interference with other variables. You write “I’d use a filehandle that wouldn’t run risk of conflict if I were putting this in a module.” Why use odd names to hopefully avoid stepping on someone when an ordinary “my” variable will do the job better?
I didn’t intend the function itself to be critiqued, but closing the file after every write is useful; it forces the file to be flushed, which means it’s visible to other processes and systems. We’re using NFS where I work, and closing files adds important functionality if I expect to be able to do a “tail -f” on them in a sensible way.
Your point about encodings is a good one, although I still don’t know the right way to solve that problem.
Perl::Critic is a tool, which can be useful or not, depending on your environment and the rules you ask it to enforce. Some people are kind of zealots about using it all the time, and can be unpleasant to deal with, and some of the things it watches for by default are not things I regard as valuable. This doesn’t make Perl::Critic itself any less useful a tool.
Perl::Critic allows you to mechanically monitor for the things described in Damian Conway’s book “Perl Best Practices”. I found that a fascinating book, and am extremely glad I read it. I agree with Mr. Conways’ premise that it’s better to understand what you’re doing than to just do things out of habit. Despite this, I disagree with many if not most of the suggestions the book makes. I have considered them and reached different conclusions.
I can’t even agree that nobody needs to have the dogma dictated to them; it’s clear from watching the world we live in that a lot of people do want to be told what is right so that they can feel they’re doing The Right Thing without having to consider it or make that decision on their own. I don’t think this is a good thing, but many clearly want it. It should come as a surprise to no one that the Perl Best Practices book created much of that same feeling. TMTOWTDI makes a lot of people really uncomfortable.
1. I don’t think that you and Tom disagree. When he said “I’d use a file handle that wouldn’t run the risk of conflict”, I thought he was suggesting a lexical scalar, *not* a typeglob with an obfuscated spelling.
2. There are (inexpensive) ways to flush a file handle without having to do an (expensive) close/open/close/open/close/open. This code is directly from “Modern Perl” pg 197\8
use autodie;
use FileHandle;
open my $fh ‘>’, ‘pecan.log’;
$fh->autoflush(1);
The subsequent text includes:
See perldoc FileHandle and perldoc IO::Handle for more information.
IO::File supersedes FileHandle in Perl 5.12
If you want to cycle the log files, and write to the ‘new’ file, you need to close the old one. Autoflush doesn’t help.
I ran into a classic case @work of why lexical fh’s are better (e.g.):
open(SOMEFILE, “> $file”) or die “Fail: $!”;
while (…) {
print SOMEFILE “$stuff\n”;
}
close SOMEFH;
system(“do_something_with $file”);
When the error was found, I pointed out the “better” way to do it which would avoid the problem in the first place, and I’ve still failed to change anyone’s habit.
Assuming at least ‘use strict’ in above example (at least “we” use that around here).
Many of Perl’s end users are more comfortable in their careers and less motivated in cleaning up their messes, and you end up with sloppy code like this example as a result. Also, a lot of them use Perl as a secondary tool to help them do sysadmin, text/data processing for academic research, etc. and only learn as much as they “need” to.
[…] to more consistently use features that have been available for a while. For example, in a recent blog post a Perl programmer talks about getting his colleagues to adopt the 3-argument form of the open() […]
[…] to more consistently use features that have been available for a while. For example, in a recent blog post a Perl programmer talks about getting his colleagues to adopt the 3-argument form of the open() […]