Thursday, March 4, 2010

Non-Reproducible? I can still fix that

How many times have you heard that an application problem is not reproducible? Yea, I know, if you're a veteran like me, way too many times and sleepless nights I bet. But non-reproducible errors are common for developers. Its just impossible most times to have development setup exactly like production when the problem was happening. There are different loads between Dev and Prod, servers and environments are typically different between them, and especially database differences, and lack of good forensic information about the problem make it hard to exactly duplicate the situation that was happening to our customers on the website. So the problems fester, we learn to live with the kinks, even put ugly band-aid solutions because its all we could find. I recall talking to a developer once at Zendcon and he told me that they created a cron scrip that would restart the apache server every night to clear up a random problem that they kept having, that was the only fix he could find. And the amount of time and effort expended by developers and the staff to troubleshoot these issues start to take a toll on the project time lines. We did a survey here at Zend and interviewed 1500 developers and found that the majority spent over 30% or more of their time with maintenance and troubleshooting production issues. Non-reproducible problems don't have to be a drain on our resources anymore, we can still find them fast (and fix them) by using new diagnostics capabilities recently released in Zend Server 5.0

One of the most talked about capabilities in the latest release of Zend Server is Code Tracing that will help address problems like these in production. Code Tracing eliminates the difficult task of reproducing the problem by automatically capturing (in real-time) the full execution history of an application problem in production. If you can imaging Zend Server continually doing a historical debugger trace of your code in real time for each request, its like a flight recorder blackbox for your PHP application constantly monitoring and storing a trace only when a problem is discovered. But what's different between Zend Server 5.0 .vs. XDEBUG trace is that your performance is not penalized for this level of diagnostics. XDEBUG will write a trace to disk for each and every line it executes generating a very large trace file and incurring a significant performance hit. Zend Server 5.0 keeps trace information in memory as it executes PHP applications and if a problem is identified through the use of error thresholds, then it will save the information to disk for you to review the problem in more details. There is even a setting to keep the the Code Trace in sleep mode, and when an error event would trigger and wake up Code Tracing. This gives you both great performance in a production environment without sacrificing deep troubleshooting abilities with Code Tracing. If you havent had a chance to see how Code Trace can help cut your own troubleshooting times in half, take a look at this Code Trade webinar and be sure to download the GA version of Zend Server 5.0 today.

Thursday, February 4, 2010

Need for Speed

Speed. We all crave it, whether it's for a faster laptop, or new phone, or latest operating system, the need for speed is an ongoing quest for us techies. So with the recent announcements from Facebook and even from us at Zend about our latest performance Benchmark, it highlights no single magic bullet or formula. Sure we can all put nitro into our cars and go really really fast for a few seconds, but in the end we will burn out our engines or have some other mechanical failure in the system. As PHP developers, we also know that speed isnt as simple as pouring nitro in a car, our systems are a lot more complex, and change a lot more often, and break a lot too. Speed is not limited by how quickly our Zend engine performs either, but also by how our database, images, browser side code, memory utilization and architecture to name just a few of the common culprits. If you read our Drupal performance benchmarking paper, you would find that in the end we were able to improve performance by over 2605% with a turbo charged version of Drupal, we even got 783% faster on the unaltered codebase, not bad for a few days work right. Well the one thing you did not read about in the whitepaper was how the bottlenecks were discovered and turbocharging accomplished. This is one of those things we rarely measure in benchmark reports or even talk about. In the industry its also known as Time to Repair, or TTR. How quickly can you identify and fix a performance problem (or for that mater, any code problem I would say is just as important in the race for application Speed and reliability)?

Sure the company cares about it. Many firms have Service Level Agreements back to there customers, but how much of our development resources and effort do you really spend trying to improve the team troubleshooting capabilities .vs. website speed?? I'd say the industry average is very very low here, and the performance diagnostics capabilities are crude at best in most PHP firms. So for the sake of time, resources and yes we throw money, and lots of it at our hardware vendor. It seems easier to upgrade our boxes then to dig into the code to try to figure out where the problems are, but what if it wasnt? What if it was easy to see it, what if we had a dashboard that points right to the problem and even told you how bad a problem it was, and what if it told you how often this performance problem occurs during peak times of business. And what if it could help you not only find bottlenecks to speedup your website, but also help with other customer issues. Would it make your company look like a rockstar because you can find and fix problems really really quickly. More speedier then your competitors do?? More cheaply then your competitors can?? This means you have better margins in your PHP business and happier customers at the same time. If you like the potential of improving speed as far as TTR, then take a look at the following diagnostics webinars, it will showcase some of the same technology our Zend experts use here. So start invest in your performance tuning and troubleshooting chops and get great customer satisfaction speed (and loyalty) for your business.

Announcing FOSE2010, #1 Gov focused Tech event of the year

As many of you know from my previous posts about Government use of PHP, I'm a big supporter of how opensource can help in the public sector including one of my 2010 predictions that this could be a great growth area this year. Well if you're like me, and want to learn more about how the government is using technology, there's an upcoming conference called FOSE in March 23-25. FOSE 2010 is the place to discover opportunities and solutions along with changing expectations for government IT professionals.

Register today for the FOSE 2010 experience: http://www.fose.com

You can expect:

- 3 days of IT resources helping you navigate today’s shifting tech landscape

- 2 full conference days packed with education on emerging technologies, trends, and new improvements to existing solutions

- Thousands of products on the FREE* EXPO floor allowing you to gain one-on-one insight into the capabilities of our exhibitors through demos, theater presentations and FREE* Education.

- Visit the Data Center and Virtualization Pavilion, CloudCamp, and the Cloud Computing Pavilion.

*FOSE is a must-attend free show for government, military, and government contractors.

It’s time to register and reserve your place at FOSE http://www.fose.com

Friday, January 8, 2010

Trends in 2010 and beyond

Its another new year and the start of the next decade for the internet and some interesting new stories have hit the net already in 2010. One very popular pastime for many in January in the blogosphere is to look into the crystal ball and predict future trends. And as with most predictions this time of year, some are just a logical progression or evolution of an idea, others are nothing short of guesstimates of the future. But in both cases it is fun to speculate on what will happen in the future and then look back and see how good a fortune teller you can be. If someone told me last week that a terrorist would board a plane and try to set off a bomb in his underwear, I would have called him crazy. But plenty of people predicted that there would be a terrorist attack on a plane this past year, that just seemed statistically probable based on prior attempts, and easy to deduce because its such a general prediction. Since I know that I have no special powers to look into the future, I will stick to logical trends I see that are a natural evolution of ideas for the PHP web in 2010.

THE INTRANET IN YOUR POCKET TREND - is a clear one for everyone to see since we have been in the middle of this trend since the first PDA’s were introduced 17yrs ago, (hint for you are a history buff - Newton was also an Apple product with limited modem connectivity back in the beginning) and now with the next gen of mobile web devices, everyone should be preparing for PHP web applications that will also fit in both form factors, PCs as well as mobile phone /tablet devices. There are far more phones then computers, and the smartphone will eclipse the PC as the dominate device to cruise the web. PHP has been a leader in this mobile space also, as some of the most popular iPhone apps come from some popular PHP sites you probably know very well. If you’ve ever used facebook or anyone of Zynga’s social games (FarmVille, Mafia Wars, Live Poker, etc.) you are already using some of the most popular iphone apps that are written with PHP backends. There are plenty of other examples, the big trend in 2010 mobile will be the explosion of enterprise apps for the iPhone and other tablets/smart phones like Google’s Android. Again most of this driven by smartphone's web browser and accessing corporate web apps built for smaller screen templates. I think PHP systems had the biggest customer growth so far on the iPhone because it was such an agile environment to take Web and Social Networking ideas and implement them quickly. For the enterprise, the mobile enable FOSS application will be helping to bring these same advantages to internal resources and information. Did you know the most popular PHP opensource blogging app Wordpress is on the iPhone already? So is Drupal, it has an iPhone template, so does Magento, SugarCRM, and Moodle, etc. This makes it easy for enterprises to make the web application ready right out of the box with these PHP applications. I predict that there will be many many more free opensource PHP apps with smartphone optimized templates that make intranet resources and content accessible to the mobile workforce. Wonder what SugarCRM would be like if it tracked field workforce via mobile GPS?, UPS is already doing something like this with its field force. This and other mobile enabled enterprise apps will be a key focus for enterprises in 2010. And yes, I know that these smartphones have a zoomable web browser for any website to work, but making it truly optimal for the small form factor to collect statistics from the road, or get reports on the road via touchscreen typing/voice lookups will require tweaking of most intranet app GUIs to be road warrior ready and useful.


MORE OPEN THEN CLOSED BUSINESS TREND – This trend has been getting a lot of press in the mobile web space lately also. As many experts are talking about the Google/Apple wars, and predicting that Google’s more open Andriod will ultimately hurt Apple’s more proprietary iPhone system. Even though I’m an Apple fanboy, I must also agree with this prediction as history has proven in the past the right mix of Open and Proprietary is the key to business success longterm. I was watching Cramer on MSNBC today who tried to tell investors that its not an either/or death match between Apple/Google, but I disagree it will ultimately shake out to one of them owning 60% or more of the smartphone market, and the others with the rest. It happened to Apple ][ that dominated personal computers until the more open IBM PC took over, then again the more closed MAC .vs. more openly available Windows on any PC. As a smart man said once, “it’s déjà vu all over again” and will happen again with smartphones. More open systems tend to compete better in free markets, and is one of the reasons most closed and proprietary firms are opening up and releasing projects based on open source. Even Microsoft has spent a lot more resources on open source initiatives this past year and on windows PHP in particular. Enterprises and Government agencies are also betting big on open initiatives in the new year, leveraging open source projects like Wiki’s, Blogs, and CMS’s inside the firewall. Some of the most popular open applications on the web are PHP apps and in 2010 I hope to see a few new ones and updates of popular ones. If last years batch is any indication of what will be released this new year, it’s going to be very exciting. I also predict new Zend Framework open source apps to be announced in 2010, it has become the most popular PHP RAD framework so I think we are at a tipping point in the community now to see the next generation of free open source apps based on it to come out. Remember the first app that was designed in the beta version of Zend Framework by IBM back in 2007, it was called QEDWIKI and other projects like Magento have since gone public to great success and acclaim. What open source Zend Framework ideas will we see in 2010 I do not know, but I talk to many internet leaders who are releasing great Zend Framework based ideas commercially every week.

BAD ECONOMY in 2010 =MORE IT AUTOMATION TREND - ok so this one is another easy trend to spot, I promises you I’m working up to my best predictions in this article like a countdown they will get more futuristic as you read on. But I do envision a continuation of automation because of the bad economy that will continue in 2010 and will make IT automation projects even more important to the bottom line. PHP is great at exchanging data as a client or a server in SOAP, REST and XMLRPC for automation of system to system communication. Webifying client/server and mini/mainframe apps will also be accelerating this year. Rapid Application Design will be a focus for most firms where PHP excels and Zend Framework /developer solutions (and other PHP frameworks) improve the productivity of your staff and quality of the web projects. Of course I’m biased to Zend’s developer suite as it’s a complete, well integrated and powerful solution for enterprise development, check out the new capabilities we just rolled out at end of 2009 and some beta videos of 2010 advances here for a look at our future versions.


UTILITY COMPUTING TREND – In a past life I worked for Loudcloud that was an early pioneer for utility computing in the data center. This was before vmware and cloud computing like xen, and the benefits to fully managed servers as a utility from a business standpoint was very compelling even back then. It was in its infancy at Loudcloud but many advances since 2000 have made managed virtualization and cloud services easier to implement, more economical and a standard in >90% of the fortune 1000 datacenters. PHP applications and architecture easily scales in this new more dynamic utility compute environment. PHP has a simpler architecture then Java app servers so its easier to implement, and scale and build dynamic web businesses in virtual and cloud environments. This is one reason VMware acquired springsource this past year as it’s a harder problem to manage and solve in Java then it is in PHP. With Zend PHP in the cloud, you will be able to take one of the oldest FOSS projects on sourceforge.org (SquirrelMail developed in 1999) and deploy it on the Amazon EC2 cloud or VMware clusters, can you do that to a Java app written in 1999 without recoding?

We here at Zend also made some new advances to ensure PHP web applications can take advantage of the specialized cloud services without locking the web app from working on only one service, so you could build a PHP app that can work across all the many cloud services and virtual datacenters without having to recode it for each. I have a few customers who have a mixed, inside outside cloud strategy and want to build a single codebase that can be deployed on internal VMware and external cloud services like Azure or EC2, but taking advantage of database service or queueing services are different in all the offerings today. And as more services are released this problem only grows for web enabled cloud computing. Our ZendPHP/partner cloud initiative is called SimpleCloud API and the software will support IBM, Microsoft, Rackspace, Amazon, Nirvanix, VMware and GoGrid services. I predict that Zend will announce new capabilities to help make management and monitoring in a cluster far easier for PHP apps. Ok, so this is more of insider information then forecast but stay tuned for our upcoming Zend Server 5.0 bundled capabilities in this space also.

SITUATIONAL APPS DRIVE GOV OSS ADOPTION TREND – New types of data are become realtime on the web. For example, today in California there was an earthquake, if you Googled it there was already references to people tweeting and blogging about it 2min after the quake. This is impressive, what other realtime data will you be able to search for in the future and what kinds of PHP ideas could be built to data mine the information that now will be available like this. Will someone create a mobile app to measure the public’s reaction to statements made by elected officials so that the message could be tweaked faster and refined. I bet Janet Napolitano could have used such an app after she made her comments that the system was working from the attempted Christmas day bombing. Maybe she would have quickly followed up in the next news briefing with a refined message if she was using such an application. This type of statistical web apps and situation management solutions are become very important to the US government, they are known as "Situational Apps" and there are a lot of PHP portals that could be used to quickly build these types of portals for government employee's to manage and respond to certain critical situations. An example would be an online work environment (think of it as a private portal/social network) used to improve US security procedures used by cross functional US agencies involved in the Christmas day bombing scenario. And the participants are both federal, local and commercial that are working together to fix this problem and what better online resource to keep these traditionally isolated groups working together, then the same technology that works out in the internet, open source PHP apps will be a key part of the solution I believe. If you viewed the QEDWIKI video earlier that's just one (non-gov) example of how PHP can build these easily, it was also the very first "Situational App Generator" that IBM created using Zend Framework and PHP back in 2007, there's been a lot progress since. Mindtouch project is a more current open source PHP based example that I like a lot for some of these also. This trend is still in its infancy in Washington, but I learned a lot about it when I attended GOSCON this past year. SO I predict that this will be a huge huge growth area for PHP business and the killer app for open source PHP adoption by the US government and other public agencies.


So let me end my article with at least one wayout 2010 prediction from the crystal ball. I predict that unlike the shoe bomber incident of last year that caused us all to have to take off our shoes at airport security, this year we will not be told to take off our underwear and place it on the conveyor for a scan. I know this is crazy talk, given how safe things have been since we all started taking our shoes off, but I fear that the underwear bomber scanning will cause more delays and possibly conceive more babies at airports if the TSA were to solve the problem the same old government way. I know, I know I really put myself out there on this last prediction, but we’ll revisit this in December 2010 and see how things shaped up.

If you have any observations you’d like to share about PHP or Zend product feature suggestions for the new year, please post them here, I’ll put some links of good posts I have read so far. Happy New Year

Thursday, December 10, 2009

Holiday Queues

Its that time of year again, Holiday’s are coming fast, and for me and other geeks we get to play with some new gadgets that we couldn’t get to all year. In the office I also use this time of year to try new things and experiment a bit because the last week in December and early January is usually a perfect time to do this for me. There’ve been some major new releases of OSS apps that I am planning to install and work with ontop of Zend Server in 2010. As my regular readers know, in the past I have already written about Zend Server and how it can significantly improve performance and identify problems in some popular OSS apps like Drupal, Joomla, Wordpress, SugarCRM and Magento.

At the holiday break I will take the latest beta released versions of phpBB 3.0x & Magento 1.4x and experiment with Zend Server’s Job Queue. When you work on a PHP project from scratch its easy to implement new capabilities, but with existing code bases like phpBB and Magento there seemed to be some very obvious places to use Job Queue and enhance performance. If you’re not familiar with Queueing, we had a good webinar this past month from Shahar the lead engineer on this. For places in the project with long-running tasks, such as order processing, catalog importing/exporting, content indexing or sending an email just to name a few obvious ones. These tasks can unnecessarily load your web server and increase application response times for the end user because the processes are sequential. Zend Server 5 beta features Job Queue support, enabling you to offload the execution of such tasks to a parallel request, or even to a different server when needed. I’m going to keep it simple and parallelize a lot of long running tasks in these projects. We have a draft of the API online already, you can find the whitepaper here for Job Queue if you want to experiment along with me.

As far as getting ready for the holiday’s, I’m am still having a hard time finding one last gift for my young nephew (who I know will be an aspiring developer/entrepreneur some day), I wanted to get him a collection of stuffed Open Source mascots as inspiration when he gets older. I already have the uber rare Mozilla, ElePHPant, Tux, but am missing and cant find a Mysql Sakila dolphin but I know some folks have a bunch like this guy in the photo, so am willing to trade PHP Elephpants for one or two and complete my stuffed animal LAMP stack. Contact me if you have any spare ones, and have a happy holiday.

Thursday, November 12, 2009

PHP at Government Open Source Conference 2009

I attended the first GOSCON 2009 (Government Open Source CONference) in DC last week and learned a lot about the government’s use of open source software (OSS). There were two major announcements at the conference, first by the DoD and the second by the White House.

The keynote with DoD CIO David Wennergren was the most anticipated event for most at this conference as his talk was about the newly released DoD guidance document that clarifies the use of open source software at the agency (and is also considered a template for all the rest.)

To quickly sum it up for anyone who missed his talk, he reiterated current federal law, which states Government agencies HAVE to look at commercial items [when building out new capabilities] and that open source is considered commercial software also. So a tip to all of us in OSS land, when dealing with the government agencies, if the agency you are working with already knows how to budget and procure commercial software today, then the procedures are the very same for agency projects that want to use open source software.” It’s that simple! then he went into specific examples in the DoD.

A lot of newspaper articles were written about how this will revolutionize the software landscape in Government, but I would argue that the revolution is already underway as most progressive government agencies are already using open source software. Though I would also say that this document will make OSS use more common place now as many in the room seemed to struggle with licensing and procurement in the past (based on the questions being asked at the conference it was just as much about OSS and the services for the project.) This memo clarifies a lot of misunderstandings within DoD (and the government) on how and when open source software can be implemented. This evens the playing field because it says open source solutions should be allowed to compete when the department is implementing a major system.

Did we really need a memo that reiterates what most enterprises have already discovered: open source saves money, is scalable, secure, and easy to adapt? Not for the early government adopters like the White House. The other big presentation at GOSCON was about the Whitehouse.gov switching its site to use open source software PHP and Drupal. For many of you who have been following my blog articles, back in February I had an article about “The most Tech Savvy use of OSS in Government” and that distinction went to BarackObama.com and the folks at Blue State Digital who took their ideas and implemented them in open source PHP to change the state of politics and campaigning forever, read more here. A little over 9months later, the president’s office does it again, and revamps one of the most important government sites, Whitehouse.gov and again proves why the most Tech Savvy president’s office uses open source PHP to make fast, flexible and frugal government a reality.

After months of planning the White House has ditched the proprietary content management system that had been in place by prior administrations in favor of the latest version of the open-source content management system based on PHP. As first reported by the AP The Obama new media team, with a few months of executive branch service under their belts, decided they needed a better development environment for the White House web presence.

After Obama’s new media team entered office and started tweaking the old WhiteHouse.gov, they decided they needed a more flexible development environment for the White House web presence. They wanted to be able to more quickly, easily, and gracefully build out their vision of interactive government. General Dynamics Information Technology (GDIT), the Virginia-based government contractor who had executed the Bush-era White House CMS contract, was tasked by the Obama Administration with finding a more flexible alternative. The ideal new platform would be one where dynamic features like question-and-answer forums, live video streaming, and collaborative tools could work more fluidly together with the site's infrastructure. The solution, says the White House, turned out to be a PHP based content management system. That's something of a victory for Drupal and PHP (not to mention open-source) community.

Is this the first move to a more open and transparent government even in code? As government developers start to embrace OSS programming and other open systems like open - forges, frameworks, languages and applications etc? I think the White House's move to open-source software signals a move towards this idea that collaborative programming can also inspire (or at least support) more open government systems.

As PHP programmers we all have seen how code openness (transparency) has made a huge impact to our own systems. When some other group/programmer wants to add or improve something to a shared OSS system, they can with open systems or they can just fork your work as a starting point. When we PHP programmers need a new system and there was already an open source project that can help get us started quickly, we just went to sourceforge.org and downloaded the PHP application to get started (just like the Whitehouse.gov site team did.) If we needed to implement something truly innovative and revolutionary, we could still use PHP to build it, even hire consultants to do it for us, and yet still we are free to build more into our system at a later date since the code is open to change (just like BarakObama.com online social politics and networking did.)

David Wheeler from DoD made a real good analogy to one attendee who was having a hard time with the concept of open code in government, he said it this plainly after several prior attempts - by comparing OSS project to buying a car for government use, "Do you want your new government car to have a hood that is welded shut because that's what proprietary software is like, or would you rather have a car that lets you open the hood, and lets you pick whatever shop or mechanic you need to improve it, not just today but into the future, and that's what open source using open languages are like" Yup that's what PHP will offer government applications, it will allow them to build IT projects with a hood that can be opened by any programmer (of course with the right approval and security - there is after all a key needed to open that hood, just like the car.)

What I think that these examples start to show us is that transparency in open code can offer transparency and agility when government polices change over time, and that’s real exciting and a huge step forward for Government. Fewer projects will be started from scratch, and more reuse and evolution of ideas will undoubtedly become the norm with more OSS proliferating government. OSS use also allows the government to reuse ideas and code that has already been proven in commercial settings. If you look at all the opportunities for different government agencies to not only reuse much of OSS code that’s out on the web, but also share more unique and US government focused projects on its own internal forges. (Forge.mil is another agency example of the forward thinking DoD open source initiatives) Many of the large enterprises that I work with have this very same type of open source policy for internal business as well as leveraging the OSS forges and applications. I’m sure this will inspire other governmental agencies (and contractors) to think about how best to leverage and share OSS code to make fast, flexible and frugal government a reality in the US. So let me know how you’re using PHP in government today and making fast, flexible and frugal government possible.

Other articles:



Monday, October 26, 2009

Zend Server 5.0 Intro to Code Tracing

Zendcon2009 ended this past week, and we introduced a new beta version of Zend Server 5.0 with several new features. In this article and video blog I will review Code Tracer. Think of it as a blackbox flight recorder for your PHP applications. When something goes wrong with an airplane, everything is saved during the flight onto the blackbox to help the investigators figure out what went wrong. What if you can do that in production with your PHP applications to figure out problems when they happen, now you can with the new Zend Server 5.0 Code Tracer.

In this video I will setup Zend Server Tracer to capture a problem from an open source application called phpExcelReader . There are two problems with this project once you download it from sourceforge that Zend Server can help you quickly identify it and fix it. I wanted to keep this first example real simple to show how to setup the CODE TRACE and used a real world FOSS application with the typical white screen problem that is so common with development. There are far more difficult problems that this feature can help you solve, including performance issues, code problems and even segfault issues when they happen a trace is the only thing that can help you figure it out. Let me know what problems this feature has helped you identify in your project, drop me a post here.

If you want to follow along be sure to get the new beta software from here:
and review the whitepaper for TRACING:


And to register for the free webinar that will go into more details and a longer demo of Code Tracer, click here.

Monday, September 14, 2009

A CTOs IT CHALLENGE - Do it in 7 Days

At Zend, I am lucky to meet and talk to a lot of innovative people who take technology such as PHP into imaginative ways in their business. I had such a conversation recently with a CTO from a large Bank who had called us for advice. During the conversation he and his team were really excited to share the details of a new PHP project that they were just about ready to release out of development, but had a problem with it. It turns out this project was the first public PHP application at the Bank, and they were typically doing everything in Java in the past. So I got curious and I asked them, What made you pick PHP for this?

The CTO said that its because PHP had proven itself to be a faster way of getting web ideas out the door. He was already a big Open Source proponent and bet big on Java at the bank. His firm had built most of its systems in Java technologies but when the projects started to pile up on his desk he got creative. He wanted to see if there was a better way to get some of this work done faster on his desk. So back in the beginning of this year, after last years books were closed, he decided to run an internal coding contest with his team. He selected one of the more common requests on his desk with a list of requirements and he sent an email out to his IT staff with a simple challenge. You have 7 days, use any technology or software in the bank even open source and proprietary applications were allowed, but you could only have three team members max on the project working on the requirements (an average size for them.) There were three submissions, one from a JAVA team who wrote everything on a Java/JBOSS platform, one from a team that used sharepoint, and a final winning team who used PHP and met most of the requirements of the project in the time alloted. They started with a popular open source PHP CMS and some custom coding to deliver on what the CTO had requested, and in turn helped him prove something important out during the contest. That it doesnt have to take months to rollout an idea or prototype at the bank. A small team can put an idea together in a just a few days to a few weeks, and then keep evolving it over time. And it doesnt have to take months to get his team upto speed either, most of his JAVA developer have now become very addicted to working on ZendFramework projects, just by initially prototyping in it. They already have many intranet PHP apps developed since the the contest. And as this most recent new project moves from the internal labs and enters public beta, the CTO was interested in sharing more details behind his early success with PHP. So an upcoming customer case study is bound to happen, but I did think his idea on how he jumpstarted PHP development at the bank (with a contest) was something that needed to be shared.

A different approach was used by another fortune 50 firm CTO I work with, he measured productivity of web projects during certain stages in the development cycle for both PHP and Java projects. Although the process used was a bit less fun then the first, the results also showed that PHP projects were more then 30% more productive then the Java way. He will be presenting the details at ZendCon.

So have you measured how much more productive PHP makes your business?, if so, drop me a line and let me know how you measure it for your firm. Also let me know what else you're doing to improve that productivity with PHP.

Monday, August 3, 2009

Web Services a la Zend

Zend Studio 7 was released this past week, and I wanted to commemorate this announcement by showing something new that hasnt been done on devzone or the rest of the web yet. In the past I have already shown how to debug a script, and how to reproduce and troubleshoot a problem, and even how to profile a bottleneck and then fix it with caching, so this time we will create a SOAP web service and use Studio to generate the WSDL.

The hardest part I find in creating SOAP services in PHP isnt the actual PHP coding but the assoc WSDL file, so in my article I will be using both Zend Studio 7 and Zend Server to create a very simple SOAP server and client and show how to easily create the WSDL file.

Let's start with the example, it will be a simple SOAP service that will just join two strings together, I'll call it JOINER. To call the SOAP service, I need a client so I put this code in a file called SOAPclient.php. (click on code to see it bigger):
Notice in the client code above, there really is only two lines 5 and 6 that are actually needed to call a SOAP service in PHP, line 5 references the WSDL file in this case it will be on local server. and on line 6 we are calling the JOINER function and passing two parameters to the service. On line 3 we are disabling WSDL caching so that if there are any changes to the file it wont be cached, you can delete that line when you are done with developement. This is all you need if you are calling a SOAP service from PHP. In our case we will also create the PHP service part and assocated WSDL file. I'll keep it all on the localhost but you all know we can move things around so that both parts are on different servers, just change the reference of "localhost" to its IP address or DNS name.

Next we need to create the service and we start with a simple PHP function that we will expose to the internet as a SOAP service. I call this file SOAPserver.php
















The first thing to note in this code is the function JOINER it looks like a normal PHP function, but to expose it to the internet as SOAP service we just need to add lines 13-15 and also create a WSDL file for it.

For the WSDL we need to add a spcial docblocks to the PHP code that specifies what TYPE the incoming and outgoing parameters will be for the service, even though PHP is loosely typed, other languages that need to talk to my SOAP service will need to know exactly what variable types to pass along, that's why we have lines 2-7 with @param and @return. This will help when we are creating the WSDL file.

Now in Zend Studio 7 to create a WSDL file, click on the SOAPservice.php file and press (CTRL+N) in menu select WEB SERVICE > WSDL

and I gave my WSDL a file name called "myservice.wsdl" see the following























Then press NEXT>


In the namespace its important to type "URN: somenamespace", and select RPC literal for a REMOTE PROCEDURE CALL. Then press FINISH button. There are 4 things to update in this next screen:

Make sure your properties tab at bottom is selected, then press on the left box called myserviceSOAP that entry inside it should say "http://example.org" you will replace this with the location of your SOAP service, in my case the following "http://localhost/MyDemos/SOAPserver.php"

Next in the center box, you should see "NewOperation" so change that to one of the function calls in our service, in our case "JOINER"

Then next to INPUT you should see "NewOperationsRequest" and I changed mine to "A" so it assoc to the first parameter in my function.

Next press the RIGHT mouse button on it and select ADD PART and change "NewPart" to "B" that is the second name of my parameter.

That's it!!! Now save your WSDL. You can look at the XML SOURCE or use the GUI WSDL designer like we did to create the file for our function.

Execute your soap service by using the client script. http://localhost/MyDemos/SOAPclient.php

If it worked you should see something like this:

SOAP WORKES joining FIRSTNAME87654321

Now for my simple example both the client and the SOAP server are on the same machine so the next step would be to move the SOAPserver.php and WSDL file to another machine and adjust the "localhost" references in both the PHP code and the WSDL file to the IP or DNS of the remote server.

It makes sense to also do this example in Zend Framework, but in the next day or two, I'll try to post a video of these same steps above atleast.





Monday, July 20, 2009

One small step for Open Source

My good buddy was celebrating the Apollo landing anniversary with his son this week with some very cool events, like building rockets, and watching the old clips of the landing, and he showed me something I didnt even know was out there. An open source computer simulator of the onboard guidance computer used in the Apollo mission. If you're a big kid, or want to introduce your own to a bit of space history, checkout this GPL simulator called Virtual AGC and AGS homepage.


Friday, May 29, 2009

Zend Server (Lesson 3) Solving a performance problem

In this Zend Server video lesson I will be dealing with a performance problem in my CMS Drupal. Performance problems happen in all sorts of scenarios and applications, so am not picking on Drupal project at all, just using it as an example in this lesson as a way to educate. Sometimes in a project, even one that's so mature like Drupal, a performance problem can be introduced just by adding in a simple tweak or new module to the core Drupal. With my site the problem is that it's sooo slow, the pages are rendering more then 5 secs with only a single user on the site, that's way too slow. I'm using the latest version of Drupal 6.12 free open source software (FOSS) code right off the site today Download Drupal 6.x and am using Zend Server + PHP5.2.9 + Zend Studio to get to the root cause of the performance problem and fix it fast. You'll soon see what the site owner did to cause this performance problem. In this video I reduce the page render time from 5sec down to 0.19sec and that's a 26 TIMES speed improvement, nice. If you want to follow along with me, review my prior lesson 1 and lesson 2 to prep for this one. And let me know what your own results are with tracking down performance issues. The lessons learned here will help you in your own PHP app, or any other PHP app on sourceforge.net

PS: I had to keep these videos short for Youtube to accept the download so its a quick 5min video lesson


Friday, May 22, 2009

Open Source helps deal with 60yr Old War Wounds

As everyone around here get's ready for another great memorial day weekend, I had a few minutes before my own break to share a WW2 story. Its about a fascinating discovery and how the internet and open source was used to solve a 60year old WW2 mystery. You see my wife never knew her grandfather growing up, he served on the USS Grunion submarine in the 2nd world war, and was MIA. But about 2yrs ago his sub was discovered and the story behind how it was discovered and the forensics was quite fascinating to anyone who loves history.

Some background the USS Grunion, was an American submarine commanded by Lieutenant Commander Mannert “Jim” Abele, disappeared on July 30, 1942, in Alaska waters. After World War II ended, Japanese records were searched but they did not reveal any mention of the sinking so the mystery of Grunion’s fate endured for decades. One day one of the sons of the captain was on the internet and bumped into some recently posted information about the Grunion on the US Sub Command site by an amateur historian from Japan.

He posted some information about a battle between the Japanese freighter Kano Maru and a US sub on the morning of July 31, 1942 near Kiska Alaska, the freighter was hit by a torpedo which knocked out the ships engines. Over the next twenty minutes three more torpedoes were fired: one passed harmlessly under the ship while the other two hit but failed to explode. The submarine surfaced and Kano Maru began firing its forecastle gun. The freighter's crew observed that the 84th shot fired hit the conning tower of sub.

After discovering information on the internet in 2002 that helped narrow down the USS Grunion’s possible location, the sons of Grunion’s commanding officer, Bruce, Brad, and John Abele, began working on a plan to find the submarine.

“This discovery has come about through a stream of seemingly improbable events; it’s like we won the lottery 10 times in a row,” said Bruce Abele, eldest son of Grunion’s commanding officer.”

I think one of the key factors that led to the successful discovery of the USS Grunion was the use of the networking effect on the internet used by the Abele brothers. In the very beginning they had setup a website to share every piece of evidence uncovered with the world. And that led to the network growing with crew family members, naval sub experts, historians, history buffs and one very important Japanese historian.

My wife discovered that the first expedition was about to go to Alaska quite by accident by doing a yahoo search for the USS Grunion and it brought up the social blogging Wordpress site (a PHP site btw.) This site became the main vehicle to communicate about the search. It was very exciting to see the search all unfold online, day after day the Abele brothers posted pictures, sonar scans and wrote in there blog to keep everyone up to date. All this hundreds of miles from the Alaska main land, in the middle of the ocean using satalite communication to update everyone back home via the internet with the latest images and details of the search. For anyone following the day by day progress, it was exciting to wake up in the morning to read what they found during these expeditions. The second one was even more action packed with HD video images of the actual sub being uploaded for all including my mother-inlaw and wife to see. It was like we were all on the boat with them.

“The synergy of our group working together with the Navy for the common cause has been a wonderful group effort,” Bruce Abele said. “The teamwork combined with everyone’s compassion and wisdom has resulted in our success.”

According to Bruce’s brother John Abele, those responsible for contributing to this discovery included historians and engineers from the U.S., Australia, Israel and Japan. Of particular note was the involvement of Japanese naval architect Yutaka Iwasaki, who provided information critical to pinpointing the location of the submarine.

The search to find all the living family members of the crew was also challenging but I think the blog also helped the family members heal those old war wounds. If you read the online blog and letters from the family members you really get a sense of how things have changed in the past 60yrs. I read some of the original letters from the crews wives/girlfriends back to the captains wife and tears would swell up. You can just feel the frustration they felt when no information was shared by the Navy, so she became the focal point and sole support contact for the 70 wives about it. Back then letters were sent back and forth between her and the crews wives.

So 60 years later those families have grown, children, grandchildren, uncles and even son-in-laws spread allover the country, and they not only write to the captains wife, but to his children and other crew families on the Wordpress Blog, and also used Skype to call Japan to thank the historian that helped start it all. I wonder what Catherine Abele would say about how things have changed to help the families cope and heal their wounds from 60yrs ago? From my wife's and her mom's point of view, the social family network that was created by the Abele brothers helped heal those old war memories. I saw it first hand and it was amazing to watch it unfold and recorded for history online.

More information about the search and history of the USS Grunion can be found in the following links. I had the pleasure to be invited to the USS Grunion Memorial in Cleveland Ohio last year and wanted to thank the Abele brothers on behalf of my wife and her family for your success in finding the USS Grunion. As many of you who read my blog already know, I worked at Netscape in the early days of the Internet, but to see firsthand how it can help solve a 60yr old WW2 mystery was amazing. Happy Memorial Day everyone, its good to remember those that made the ultimate sacrifice and to thank their surviving families.

Read more: "The Search for the USS Grunion: How a Missing United States Submarine was Found After Six Decades


http://www.USSGrunion.com/


US Navy Press Release regarding USS Grunion (SS-216) discovery

Eternal Patrol The Fate of the USS Grunion and the Search article


The Search for the USS Grunion
with HD images

USAToday Article, click Exploring the Vessel for video of the discovery
at the top of article

TODAY show piece about the search and persistence of the Abele brothers:


Most Savvy use of Open Source in Gov awarded

I attended the April Boston PHP User group meeting to give the ELE award for the most savvy use of Open Source to the team at Blue State Digital that built BarackObama.com. If you recall back in Feb we announced the winners in this post, here is the video presentation from the April meeting and a snap of the awards given by me. Congratulations to all. Josh King and Chuck Hagenbuch share their experiences building and scaling the tools which powered my.barackobama.com and the millions of phone calls and billion emails that helped make history on January 20th, 2009: http://www.bostonphp.org/content/view/130/1/

Friday, May 1, 2009

Taking Zend Server for a Spin - (Part 2 Troubleshooting)

In my last article we took Zend Server for a quick spin, getting some Zend Framework demos to work and we started to look at application monitoring. In this article I will be configuring Debugging/Profiling between Zend Server and Zend Studio Eclipse to enable better troubleshooting and quicker resolution. When things go wrong in your app, you want to know about it ASAP and fix the problem quickly before it affects the end-user. Zend Studio Eclipse can be tightly integrated with Zend Server and gives you a powerful way to address application problems. I'll be building upon my last article so if you want to follow along, please install Zend Server and get everything working from my PART 1.

Zend Studio Eclipse (ZSE) by itself supports local debugging out of the box, when you install ZSE, it installs its own PHP environment and allows you to "Debug Scripts". The downside is that the bundled Zend Studio PHP environment may not have all the extensions and capabilities you need to replicate your production/dev environment. This is why Zend Studio also supports "Remote Debugging", and dont be confused by the name, since you can use Remote debugging on the same server. Think of it as Client/Server debugging. The Client is Zend Studio that can debug against any other Server side PHP with the appropriate Zend Debugger installed. In this article I will be using Zend Server with the preinstalled debugger to get all this working.


Steps That I will show in my Video

1) Pre-req to install Zend Server and Zend Studio Eclipse
2) Verify debugger is active on Zend Server
3) Configure debugging permission for the Zend Server
4) Browse website page and Debug from IE or FF toolbar
5) Test Debugging in Zend Eclipse
6) Test Event Debugging and Troubleshooting from Zend Server
7) Test Profiling of an application bottleneck




Tuesday, April 7, 2009

New Zend Application Server released today

It was an exciting day here at Zend today, we officially released Zend Server into GA, and I have to say this is an important milestone for the industry. It reminds me of the time 12 yrs ago when I was working on another new server technology at Netscape the very first Java Application Server that started a whole new category of software and opportunities on the web. Well NSCP's app server was technically not the first version as NSCP merged with the first one built by Kivasoft and release the next version under the new NSCP name. But it was that next version that made a huge impact on the internet and the name change was one of those things that signaled a maturing of the corprate market that was ready to adopt application servers as solutions. Websites needed better Scalability, High Performance and productivity improvements over the old way of doing things on the internet back then.

NSCP App Server was the beginning of the shift away from client/server corporate initiatives and desktop apps development like powerbuilder and visual studio onto the web. As of today only 33% of these internal employee applications have moved in 12yrs to the web, and that's mostly because of the lower productivity and high startup costs of early app server solutions. Scripting languages like PHP improve things at the language layer, but a complete solution is needed to addresses best practices and gain even more productivity while developing your next web ideas. As Yogi said once "This is like deja vu all over again"

Now fast forward to today's release of Zend Server, it offers PHP websites enterprise capabilities without the complexity and overhead of those first generation Java App Servers. Sure it can help with High Performance, it has the fastest PHP Byte Code accelerator technology out there. No other PHP caching technology is faster either, not even APC, and if you are running PHP applications on Windows then there is no other technology that is more reliable and tested for production environments under stress then Zend Server. We've implemented an APC function mapping layer into Zend Server that will allow your PHP apps using APC calls to utilize the faster internals of Zend Server without need to change your app code. So examples like phpBB modified to use it will run on Zend Server without code changes, and run even faster but with additional capabilities that will surprise you. Like better troubleshooting.

Quickly pinpointing the root cause of a problem and the tools to reproduce them faster is a major feature developers are surprised about when they start to use Zend Server. Like a sports car that you test drive because it performs well on the road, you the driver soon fall in love with the internals. As you sit behind that wheel you'll really get a good feel of the Zend engine and quick acceleration, but also fall in love with all the other things that makes driving it special. So take the new Zend Server out for a spin, you will also find many surprises that will WOW you even more then the thrill for speed as you sit behind the wheel. I'll be blogging more about those in the next few weeks, but since we are on the topic of speed. I had a meeting with one of our closest customers today, and used Zend Page caching to make a new PHP app more then 70 times faster, took only a few min after installing the server. I'll show you how to do the same thing in opensource using Joomla and Drupal in my upcoming article.

Thursday, April 2, 2009

Zend Framework can save your weekends?

And maybe even your marriage. I wrote this on Apr 2nd so that its not mistaken for an April fools joke, but a customer of ours wrote a very interesting article about the benefits he sees from using Zend Framework. He listed many great benefits of ZF standardization at Panasonic, but one of the more interesting ones was a business value that I have not seen blogged about much.
Jim Plush wrote: Upsides to this for (my) developers is the fact you're getting trained on the most popular php framework out there and no longer will you have to spend your weekends learning someone (else's) code who just quite.

This indeed is a major problem faced by many managers who have to support not just PHP, but other legacy environments both old PHP ones and other apps like the one Jim writes about in his blog when the team changes over time. As managers we know that successful apps will live on for many many years so a good strategy of how to reduce support costs over time was an important factor for Jim, so his solution was to simplify and standardize on best practices with the zend framework moving forward. As he put it so well in his own words.

We're dedicating all in house development to Zend Framework. If it's a web app it's either built in our Zend Framework infrastructure or it doesn't get built (in Panasonic.)


This final statement was something I have been hearing a lot from development managers who are optimizing the productivity of their web development efforts using PHP. I hope to share this same scenario from one of the largest Fortune 500 firms in an upcoming article. Today I learned that they also are writing about their own results using Zend so I wont spoil the presentation. Stay tuned. But if you want to learn more about why Jim at Panasonic chose Zend Framework and PHP for his Enterprise, read on.

It looks like he's also looking for a few more good people to add to his Zend Framework team, read job post here - the March job numbers are expected to be horrible, but folks with Zend skills are doing very well indeed this year.

Wednesday, April 1, 2009

Taking Zend Server for a Spin (part 1)

April in my neck of the woods is when the weather starts to get nice and Spring is in full bloom, its also the time my wife and I take our classic car out for a spin. Its fun to drive with the top down to shake off winter blues and drive somewhere new. Well I've been doing a lot of that at work too testing the new Zend Server and seeing what's under the hood (not just the Zend Engine 2, but all the other things that make the new server go).

If you've missed all the news and announcements about Zend Server, take a look at Andi's blog, it has some real good posts and an insiders look at the Windows and Linux versions.

In my blog this week I'd like to take Zend Server for a spin, if you want to follow along you will need to install it first, and then activate all features by installing a trial license, your first screen should look all green on the right side like the figure above. If not, you may not have licensed your trial copy, click here for instructions, or if you originally decided not to active some features, its easy to reactivate.

Next let's setup some PHP apps to work with, setting up the preinstalled Zend Framework demos are easy to do. If IIS is your web server I recommend using IIS7 that supports the apache rewrite rules module and just copy the demo zend framework directory into the www root.

If youre using apache web server you can also copy the "demo" directory from the Zend Framework install, to your htdocs web root. To find out where the demo's are located, you can do a browser page search in the following PHPINFO screen, it can be found in ZendServer Admin menu Monitor>PHP Info and then do a search in the page for the word ZendFramework. That will list out the entire path downto the library subdirectory, just go up one level and you will see the demo folder.

If you copyed it in the right place it should all work now, your first Zend Framework demo should display now in the browser, http://localhost/Demo/Gdata/YouTubeVideoBrowser and it should look like this screen (BTW you can click on any screen shot to zoom in and get a bigger view of it):

This Youtube demo is using the Zend Framework GDATA modules, there are many other examples that interface into Google apps like calendar, spreadsheet, photos, health, books, docs, and blogger. For full details of the ZF API for Google apps and all source is provided on how to do it in PHP. This would be a great way to build some of those web apps that I wrote about from Genentech last week.

There are quite a few web services examples, to run the Amazon book example type the following,
http://localhost/Demo/WebServices/Amazon/amazon-search.php there is also a UPC lookup example http://localhost/Demo/WebServices/Protocols/xmlrpc-upc-lookup.php

If you try to run the flickr demo you will probably only see a white screen in the browser come up.
http://localhost/Demo/WebServices/Flickr/flickr-search.php Is there something wrong with the demo or code bug? To help figure this out, let's go back to our Zend Server monitor screen and look at the latest error at the top of the screen, I highlighted it in grey here.Click on this first event to get even more details about the problem we are having. Click the ERROR DATA tab in the middle of the screen and we can see exactly why the problem is happening, its an INVALID API KEY.
So let us click Show File in Zend Server if you have that up and running and enter a valid key in the file flickr-search.php, you should register for your own key, its free at flickr and can be found here. There are a lot of problems scenarios like this that come up in real production and testing environments, that makes Zend Server ideal to track down quickly. Many times just looking at all the data collected is enough to figure out the problem like in this case, other times the applicaiton and problem is more complex and we would need to troubleshoot it with the Zend remote debugger or profiler. I'll have some more examples that will show how this is used, but for now let us just edit the file in Zend Studio and enter the API key in the code.

And save it, to test it goto http://localhost/Demo/WebServices/Flickr/flickr-search.php and if your key is valid you'll get a screen full of pictures instead of a blank window as before.
Next let us look at a feed demo, its a typical thing you would find in any content management system or PHP portal like Joomla, Drupal, or Dekiwiki. http://localhost/Demo/Zend/Feeds/consume-feed.php If you go back to Zend Server you will notice a new event at the top "SEVERE SLOW REQUEST EXECUTION", that's a bottleneck identified by Zend Server. On my laptop it sometime takes more then 2sec to execute this feed, I should know because on my firefox I have also installed a plugin to show me the rendering time of a page (FF plugin is called LORI)

There's another useful utility called the Zend Controller preinstalled in the bin subdirectory of ZendServer that can help us measure performance. Let's paste the feed URL into the utility and run a quick test to see how many requests per sec it can handle. Looks like only 1.1 requests/sec on my laptop, see figure below:

Now lets use Zend Server's page level caching to improve the performance of this feed as you can imagine this is a common problem in many portals and wiki's. I go back to Zend Server Admin, menu>Rules Management>Caching and will add a new caching rule.

I called my rule "FASTFEED" I filled in the URL to cache the output "http://localhost/Demo/Zend/Feeds/consume-feed.php" and then set the rule to cache the output for 360 sec and saved my script. Be sure to RESTART PHP after saving rule for it to take effect. Now I re-run my performance benchmark in Zend Controller, here's a screen shot of both the feed and performance test:



Notice in the Zend Controller I now have 3.75 times more requests per sec, and in Firefox the page speed went from 2-0.9/sec to 0.24/sec a dramatic speed improvement. Since this is a RSS feed, the timings vary wildly at different times during the day but this example is representative of realworld situations. Also note the Zend Controller measures first byte returned so tools like Apache Benchmark, JMETER and LOADRUNNER are good tools for further benchmarking.

So download Zend Server and take it for a spin, if you also use open source projects like SugarCRM for example, many will detect the installation of Zend Server and its data caching and will perform significantly faster automatically. In an upcoming article, I'll take some of those for a spin too and list out all the ones that I have tested. For more news on Zend Server release, click here.

The band I've been listening to the most while taking things for a spin this month is
Kings of Leon