PHP at Government Open Source Conference 2009

I attended the first GOSCON 2009 (Government Open Source CONference) in DC last week and learned a lot about the government’s use of open source software (OSS). There were two major announcements at the conference, first by the DoD and the second by the White House.

The keynote with DoD CIO David Wennergren was the most anticipated event for most at this conference as his talk was about the newly released DoD guidance document that clarifies the use of open source software at the agency (and is also considered a template for all the rest.)

To quickly sum it up for anyone who missed his talk, he reiterated current federal law, which states Government agencies HAVE to look at commercial items [when building out new capabilities] and that open source is considered commercial software also. So a tip to all of us in OSS land, when dealing with the government agencies, if the agency you are working with already knows how to budget and procure commercial software today, then the procedures are the very same for agency projects that want to use open source software.” It’s that simple! then he went into specific examples in the DoD.

A lot of newspaper articles were written about how this will revolutionize the software landscape in Government, but I would argue that the revolution is already underway as most progressive government agencies are already using open source software. Though I would also say that this document will make OSS use more common place now as many in the room seemed to struggle with licensing and procurement in the past (based on the questions being asked at the conference it was just as much about OSS and the services for the project.) This memo clarifies a lot of misunderstandings within DoD (and the government) on how and when open source software can be implemented. This evens the playing field because it says open source solutions should be allowed to compete when the department is implementing a major system.

Did we really need a memo that reiterates what most enterprises have already discovered: open source saves money, is scalable, secure, and easy to adapt? Not for the early government adopters like the White House. The other big presentation at GOSCON was about the Whitehouse.gov switching its site to use open source software PHP and Drupal. For many of you who have been following my blog articles, back in February I had an article about “The most Tech Savvy use of OSS in Government” and that distinction went to BarackObama.com and the folks at Blue State Digital who took their ideas and implemented them in open source PHP to change the state of politics and campaigning forever, read more here. A little over 9months later, the president’s office does it again, and revamps one of the most important government sites, Whitehouse.gov and again proves why the most Tech Savvy president’s office uses open source PHP to make fast, flexible and frugal government a reality.

After months of planning the White House has ditched the proprietary content management system that had been in place by prior administrations in favor of the latest version of the open-source content management system based on PHP. As first reported by the AP The Obama new media team, with a few months of executive branch service under their belts, decided they needed a better development environment for the White House web presence.

After Obama’s new media team entered office and started tweaking the old WhiteHouse.gov, they decided they needed a more flexible development environment for the White House web presence. They wanted to be able to more quickly, easily, and gracefully build out their vision of interactive government. General Dynamics Information Technology (GDIT), the Virginia-based government contractor who had executed the Bush-era White House CMS contract, was tasked by the Obama Administration with finding a more flexible alternative. The ideal new platform would be one where dynamic features like question-and-answer forums, live video streaming, and collaborative tools could work more fluidly together with the site's infrastructure. The solution, says the White House, turned out to be a PHP based content management system. That's something of a victory for Drupal and PHP (not to mention open-source) community.

Is this the first move to a more open and transparent government even in code? As government developers start to embrace OSS programming and other open systems like open - forges, frameworks, languages and applications etc? I think the White House's move to open-source software signals a move towards this idea that collaborative programming can also inspire (or at least support) more open government systems.

As PHP programmers we all have seen how code openness (transparency) has made a huge impact to our own systems. When some other group/programmer wants to add or improve something to a shared OSS system, they can with open systems or they can just fork your work as a starting point. When we PHP programmers need a new system and there was already an open source project that can help get us started quickly, we just went to sourceforge.org and downloaded the PHP application to get started (just like the Whitehouse.gov site team did.) If we needed to implement something truly innovative and revolutionary, we could still use PHP to build it, even hire consultants to do it for us, and yet still we are free to build more into our system at a later date since the code is open to change (just like BarakObama.com online social politics and networking did.)

David Wheeler from DoD made a real good analogy to one attendee who was having a hard time with the concept of open code in government, he said it this plainly after several prior attempts - by comparing OSS project to buying a car for government use, "Do you want your new government car to have a hood that is welded shut because that's what proprietary software is like, or would you rather have a car that lets you open the hood, and lets you pick whatever shop or mechanic you need to improve it, not just today but into the future, and that's what open source using open languages are like" Yup that's what PHP will offer government applications, it will allow them to build IT projects with a hood that can be opened by any programmer (of course with the right approval and security - there is after all a key needed to open that hood, just like the car.)

What I think that these examples start to show us is that transparency in open code can offer transparency and agility when government polices change over time, and that’s real exciting and a huge step forward for Government. Fewer projects will be started from scratch, and more reuse and evolution of ideas will undoubtedly become the norm with more OSS proliferating government. OSS use also allows the government to reuse ideas and code that has already been proven in commercial settings. If you look at all the opportunities for different government agencies to not only reuse much of OSS code that’s out on the web, but also share more unique and US government focused projects on its own internal forges. (Forge.mil is another agency example of the forward thinking DoD open source initiatives) Many of the large enterprises that I work with have this very same type of open source policy for internal business as well as leveraging the OSS forges and applications. I’m sure this will inspire other governmental agencies (and contractors) to think about how best to leverage and share OSS code to make fast, flexible and frugal government a reality in the US. So let me know how you’re using PHP in government today and making fast, flexible and frugal government possible.

Other articles:

http://personaldemocracy.com/node/15131

http://buytaert.net/whitehouse-gov-using-drupal

http://radar.oreilly.com/2009/10/whitehouse-switch-drupal-opensource.html

Zend Server 5.0 Intro to Code Tracing

Zendcon2009 ended this past week, and we introduced a new beta version of Zend Server 5.0 with several new features. In this article and video blog I will review Code Tracer. Think of it as a blackbox flight recorder for your PHP applications. When something goes wrong with an airplane, everything is saved during the flight onto the blackbox to help the investigators figure out what went wrong. What if you can do that in production with your PHP applications to figure out problems when they happen, now you can with the new Zend Server 5.0 Code Tracer.

In this video I will setup Zend Server Tracer to capture a problem from an open source application called phpExcelReader . There are two problems with this project once you download it from sourceforge that Zend Server can help you quickly identify it and fix it. I wanted to keep this first example real simple to show how to setup the CODE TRACE and used a real world FOSS application with the typical white screen problem that is so common with development. There are far more difficult problems that this feature can help you solve, including performance issues, code problems and even segfault issues when they happen a trace is the only thing that can help you figure it out. Let me know what problems this feature has helped you identify in your project, drop me a post here.

If you want to follow along be sure to get the new beta software from here:

http://forums.zend.com/viewtopic.php?f=74&t=3793

and review the whitepaper for TRACING:

http://forums.zend.com/viewforum.php?f=74

And to register for the free webinar that will go into more details and a longer demo of Code Tracer, click here.

A CTOs IT CHALLENGE - Do it in 7 Days

At Zend, I am lucky to meet and talk to a lot of innovative people who take technology such as PHP into imaginative ways in their business. I had such a conversation recently with a CTO from a large Bank who had called us for advice. During the conversation he and his team were really excited to share the details of a new PHP project that they were just about ready to release out of development, but had a problem with it. It turns out this project was the first public PHP application at the Bank, and they were typically doing everything in Java in the past. So I got curious and I asked them, What made you pick PHP for this?

The CTO said that its because PHP had proven itself to be a faster way of getting web ideas out the door. He was already a big Open Source proponent and bet big on Java at the bank. His firm had built most of its systems in Java technologies but when the projects started to pile up on his desk he got creative. He wanted to see if there was a better way to get some of this work done faster on his desk. So back in the beginning of this year, after last years books were closed, he decided to run an internal coding contest with his team. He selected one of the more common requests on his desk with a list of requirements and he sent an email out to his IT staff with a simple challenge. You have 7 days, use any technology or software in the bank even open source and proprietary applications were allowed, but you could only have three team members max on the project working on the requirements (an average size for them.) There were three submissions, one from a JAVA team who wrote everything on a Java/JBOSS platform, one from a team that used sharepoint, and a final winning team who used PHP and met most of the requirements of the project in the time alloted. They started with a popular open source PHP CMS and some custom coding to deliver on what the CTO had requested, and in turn helped him prove something important out during the contest. That it doesnt have to take months to rollout an idea or prototype at the bank. A small team can put an idea together in a just a few days to a few weeks, and then keep evolving it over time. And it doesnt have to take months to get his team upto speed either, most of his JAVA developer have now become very addicted to working on ZendFramework projects, just by initially prototyping in it. They already have many intranet PHP apps developed since the the contest. And as this most recent new project moves from the internal labs and enters public beta, the CTO was interested in sharing more details behind his early success with PHP. So an upcoming customer case study is bound to happen, but I did think his idea on how he jumpstarted PHP development at the bank (with a contest) was something that needed to be shared.

A different approach was used by another fortune 50 firm CTO I work with, he measured productivity of web projects during certain stages in the development cycle for both PHP and Java projects. Although the process used was a bit less fun then the first, the results also showed that PHP projects were more then 30% more productive then the Java way. He will be presenting the details at ZendCon.

So have you measured how much more productive PHP makes your business?, if so, drop me a line and let me know how you measure it for your firm. Also let me know what else you're doing to improve that productivity with PHP.

Web Services a la Zend

Zend Studio 7 was released this past week, and I wanted to commemorate this announcement by showing something new that hasnt been done on devzone or the rest of the web yet. In the past I have already shown how to debug a script, and how to reproduce and troubleshoot a problem, and even how to profile a bottleneck and then fix it with caching, so this time we will create a SOAP web service and use Studio to generate the WSDL.

The hardest part I find in creating SOAP services in PHP isnt the actual PHP coding but the assoc WSDL file, so in my article I will be using both Zend Studio 7 and Zend Server to create a very simple SOAP server and client and show how to easily create the WSDL file.

Let's start with the example, it will be a simple SOAP service that will just join two strings together, I'll call it JOINER. To call the SOAP service, I need a client so I put this code in a file called SOAPclient.php. (click on code to see it bigger):
Notice in the client code above, there really is only two lines 5 and 6 that are actually needed to call a SOAP service in PHP, line 5 references the WSDL file in this case it will be on local server. and on line 6 we are calling the JOINER function and passing two parameters to the service. On line 3 we are disabling WSDL caching so that if there are any changes to the file it wont be cached, you can delete that line when you are done with developement. This is all you need if you are calling a SOAP service from PHP. In our case we will also create the PHP service part and assocated WSDL file. I'll keep it all on the localhost but you all know we can move things around so that both parts are on different servers, just change the reference of "localhost" to its IP address or DNS name.

Next we need to create the service and we start with a simple PHP function that we will expose to the internet as a SOAP service. I call this file SOAPserver.php
















The first thing to note in this code is the function JOINER it looks like a normal PHP function, but to expose it to the internet as SOAP service we just need to add lines 13-15 and also create a WSDL file for it.

For the WSDL we need to add a spcial docblocks to the PHP code that specifies what TYPE the incoming and outgoing parameters will be for the service, even though PHP is loosely typed, other languages that need to talk to my SOAP service will need to know exactly what variable types to pass along, that's why we have lines 2-7 with @param and @return. This will help when we are creating the WSDL file.

Now in Zend Studio 7 to create a WSDL file, click on the SOAPservice.php file and press (CTRL+N) in menu select WEB SERVICE > WSDL

and I gave my WSDL a file name called "myservice.wsdl" see the following

Then press NEXT>


In the namespace its important to type "URN: somenamespace", and select RPC literal for a REMOTE PROCEDURE CALL. Then press FINISH button. There are 4 things to update in this next screen:

Make sure your properties tab at bottom is selected, then press on the left box called myserviceSOAP that entry inside it should say "http://example.org" you will replace this with the location of your SOAP service, in my case the following "http://localhost/MyDemos/SOAPserver.php"

Next in the center box, you should see "NewOperation" so change that to one of the function calls in our service, in our case "JOINER"

Then next to INPUT you should see "NewOperationsRequest" and I changed mine to "A" so it assoc to the first parameter in my function.

Next press the RIGHT mouse button on it and select ADD PART and change "NewPart" to "B" that is the second name of my parameter.

That's it!!! Now save your WSDL. You can look at the XML SOURCE or use the GUI WSDL designer like we did to create the file for our function.

Execute your soap service by using the client script. http://localhost/MyDemos/SOAPclient.php

If it worked you should see something like this:

SOAP WORKES joining FIRSTNAME87654321

Now for my simple example both the client and the SOAP server are on the same machine so the next step would be to move the SOAPserver.php and WSDL file to another machine and adjust the "localhost" references in both the PHP code and the WSDL file to the IP or DNS of the remote server.

It makes sense to also do this example in Zend Framework, but in the next day or two, I'll try to post a video of these same steps above atleast.

One small step for Open Source

My good buddy was celebrating the Apollo landing anniversary with his son this week with some very cool events, like building rockets, and watching the old clips of the landing, and he showed me something I didnt even know was out there. An open source computer simulator of the onboard guidance computer used in the Apollo mission. If you're a big kid, or want to introduce your own to a bit of space history, checkout this GPL simulator called Virtual AGC and AGS homepage.

Zend Server (Lesson 3) Solving a performance problem

In this Zend Server video lesson I will be dealing with a performance problem in my CMS Drupal. Performance problems happen in all sorts of scenarios and applications, so am not picking on Drupal project at all, just using it as an example in this lesson as a way to educate. Sometimes in a project, even one that's so mature like Drupal, a performance problem can be introduced just by adding in a simple tweak or new module to the core Drupal. With my site the problem is that it's sooo slow, the pages are rendering more then 5 secs with only a single user on the site, that's way too slow. I'm using the latest version of Drupal 6.12 free open source software (FOSS) code right off the site today Download Drupal 6.x and am using Zend Server + PHP5.2.9 + Zend Studio to get to the root cause of the performance problem and fix it fast. You'll soon see what the site owner did to cause this performance problem. In this video I reduce the page render time from 5sec down to 0.19sec and that's a 26 TIMES speed improvement, nice. If you want to follow along with me, review my prior lesson 1 and lesson 2 to prep for this one. And let me know what your own results are with tracking down performance issues. The lessons learned here will help you in your own PHP app, or any other PHP app on sourceforge.net

PS: I had to keep these videos short for Youtube to accept the download so its a quick 5min video lesson

Open Source helps deal with 60yr Old War Wounds

As everyone around here get's ready for another great memorial day weekend, I had a few minutes before my own break to share a WW2 story. Its about a fascinating discovery and how the internet and open source was used to solve a 60year old WW2 mystery. You see my wife never knew her grandfather growing up, he served on the USS Grunion submarine in the 2nd world war, and was MIA. But about 2yrs ago his sub was discovered and the story behind how it was discovered and the forensics was quite fascinating to anyone who loves history.

Some background the USS Grunion, was an American submarine commanded by Lieutenant Commander Mannert “Jim” Abele, disappeared on July 30, 1942, in Alaska waters. After World War II ended, Japanese records were searched but they did not reveal any mention of the sinking so the mystery of Grunion’s fate endured for decades. One day one of the sons of the captain was on the internet and bumped into some recently posted information about the Grunion on the US Sub Command site by an amateur historian from Japan.

He posted some information about a battle between the Japanese freighter Kano Maru and a US sub on the morning of July 31, 1942 near Kiska Alaska, the freighter was hit by a torpedo which knocked out the ships engines. Over the next twenty minutes three more torpedoes were fired: one passed harmlessly under the ship while the other two hit but failed to explode. The submarine surfaced and Kano Maru began firing its forecastle gun. The freighter's crew observed that the 84th shot fired hit the conning tower of sub.

After discovering information on the internet in 2002 that helped narrow down the USS Grunion’s possible location, the sons of Grunion’s commanding officer, Bruce, Brad, and John Abele, began working on a plan to find the submarine.

“This discovery has come about through a stream of seemingly improbable events; it’s like we won the lottery 10 times in a row,” said Bruce Abele, eldest son of Grunion’s commanding officer.”

I think one of the key factors that led to the successful discovery of the USS Grunion was the use of the networking effect on the internet used by the Abele brothers. In the very beginning they had setup a website to share every piece of evidence uncovered with the world. And that led to the network growing with crew family members, naval sub experts, historians, history buffs and one very important Japanese historian.

My wife discovered that the first expedition was about to go to Alaska quite by accident by doing a yahoo search for the USS Grunion and it brought up the social blogging Wordpress site (a PHP site btw.) This site became the main vehicle to communicate about the search. It was very exciting to see the search all unfold online, day after day the Abele brothers posted pictures, sonar scans and wrote in there blog to keep everyone up to date. All this hundreds of miles from the Alaska main land, in the middle of the ocean using satalite communication to update everyone back home via the internet with the latest images and details of the search. For anyone following the day by day progress, it was exciting to wake up in the morning to read what they found during these expeditions. The second one was even more action packed with HD video images of the actual sub being uploaded for all including my mother-inlaw and wife to see. It was like we were all on the boat with them.

“The synergy of our group working together with the Navy for the common cause has been a wonderful group effort,” Bruce Abele said. “The teamwork combined with everyone’s compassion and wisdom has resulted in our success.”

According to Bruce’s brother John Abele, those responsible for contributing to this discovery included historians and engineers from the U.S., Australia, Israel and Japan. Of particular note was the involvement of Japanese naval architect Yutaka Iwasaki, who provided information critical to pinpointing the location of the submarine.

The search to find all the living family members of the crew was also challenging but I think the blog also helped the family members heal those old war wounds. If you read the online blog and letters from the family members you really get a sense of how things have changed in the past 60yrs. I read some of the original letters from the crews wives/girlfriends back to the captains wife and tears would swell up. You can just feel the frustration they felt when no information was shared by the Navy, so she became the focal point and sole support contact for the 70 wives about it. Back then letters were sent back and forth between her and the crews wives.

So 60 years later those families have grown, children, grandchildren, uncles and even son-in-laws spread allover the country, and they not only write to the captains wife, but to his children and other crew families on the Wordpress Blog, and also used Skype to call Japan to thank the historian that helped start it all. I wonder what Catherine Abele would say about how things have changed to help the families cope and heal their wounds from 60yrs ago? From my wife's and her mom's point of view, the social family network that was created by the Abele brothers helped heal those old war memories. I saw it first hand and it was amazing to watch it unfold and recorded for history online.

More information about the search and history of the USS Grunion can be found in the following links. I had the pleasure to be invited to the USS Grunion Memorial in Cleveland Ohio last year and wanted to thank the Abele brothers on behalf of my wife and her family for your success in finding the USS Grunion. As many of you who read my blog already know, I worked at Netscape in the early days of the Internet, but to see firsthand how it can help solve a 60yr old WW2 mystery was amazing. Happy Memorial Day everyone, its good to remember those that made the ultimate sacrifice and to thank their surviving families.

Read more: "The Search for the USS Grunion: How a Missing United States Submarine was Found After Six Decades


http://www.USSGrunion.com/

US Navy Press Release regarding USS Grunion (SS-216) discovery

Eternal Patrol The Fate of the USS Grunion and the Search article


The Search for the USS Grunion with HD images

USAToday Article, click Exploring the Vessel for video of the discovery at the top of article

TODAY show piece about the search and persistence of the Abele brothers: