Thursday, March 4, 2010

Non-Reproducible? I can still fix that

How many times have you heard that an application problem is not reproducible? Yea, I know, if you're a veteran like me, way too many times and sleepless nights I bet. But non-reproducible errors are common for developers. Its just impossible most times to have development setup exactly like production when the problem was happening. There are different loads between Dev and Prod, servers and environments are typically different between them, and especially database differences, and lack of good forensic information about the problem make it hard to exactly duplicate the situation that was happening to our customers on the website. So the problems fester, we learn to live with the kinks, even put ugly band-aid solutions because its all we could find. I recall talking to a developer once at Zendcon and he told me that they created a cron scrip that would restart the apache server every night to clear up a random problem that they kept having, that was the only fix he could find. And the amount of time and effort expended by developers and the staff to troubleshoot these issues start to take a toll on the project time lines. We did a survey here at Zend and interviewed 1500 developers and found that the majority spent over 30% or more of their time with maintenance and troubleshooting production issues. Non-reproducible problems don't have to be a drain on our resources anymore, we can still find them fast (and fix them) by using new diagnostics capabilities recently released in Zend Server 5.0

One of the most talked about capabilities in the latest release of Zend Server is Code Tracing that will help address problems like these in production. Code Tracing eliminates the difficult task of reproducing the problem by automatically capturing (in real-time) the full execution history of an application problem in production. If you can imaging Zend Server continually doing a historical debugger trace of your code in real time for each request, its like a flight recorder blackbox for your PHP application constantly monitoring and storing a trace only when a problem is discovered. But what's different between Zend Server 5.0 .vs. XDEBUG trace is that your performance is not penalized for this level of diagnostics. XDEBUG will write a trace to disk for each and every line it executes generating a very large trace file and incurring a significant performance hit. Zend Server 5.0 keeps trace information in memory as it executes PHP applications and if a problem is identified through the use of error thresholds, then it will save the information to disk for you to review the problem in more details. There is even a setting to keep the the Code Trace in sleep mode, and when an error event would trigger and wake up Code Tracing. This gives you both great performance in a production environment without sacrificing deep troubleshooting abilities with Code Tracing. If you havent had a chance to see how Code Trace can help cut your own troubleshooting times in half, take a look at this Code Trade webinar and be sure to download the GA version of Zend Server 5.0 today.

Thursday, February 4, 2010

Need for Speed

Speed. We all crave it, whether it's for a faster laptop, or new phone, or latest operating system, the need for speed is an ongoing quest for us techies. So with the recent announcements from Facebook and even from us at Zend about our latest performance Benchmark, it highlights no single magic bullet or formula. Sure we can all put nitro into our cars and go really really fast for a few seconds, but in the end we will burn out our engines or have some other mechanical failure in the system. As PHP developers, we also know that speed isnt as simple as pouring nitro in a car, our systems are a lot more complex, and change a lot more often, and break a lot too. Speed is not limited by how quickly our Zend engine performs either, but also by how our database, images, browser side code, memory utilization and architecture to name just a few of the common culprits. If you read our Drupal performance benchmarking paper, you would find that in the end we were able to improve performance by over 2605% with a turbo charged version of Drupal, we even got 783% faster on the unaltered codebase, not bad for a few days work right. Well the one thing you did not read about in the whitepaper was how the bottlenecks were discovered and turbocharging accomplished. This is one of those things we rarely measure in benchmark reports or even talk about. In the industry its also known as Time to Repair, or TTR. How quickly can you identify and fix a performance problem (or for that mater, any code problem I would say is just as important in the race for application Speed and reliability)?

Sure the company cares about it. Many firms have Service Level Agreements back to there customers, but how much of our development resources and effort do you really spend trying to improve the team troubleshooting capabilities .vs. website speed?? I'd say the industry average is very very low here, and the performance diagnostics capabilities are crude at best in most PHP firms. So for the sake of time, resources and yes we throw money, and lots of it at our hardware vendor. It seems easier to upgrade our boxes then to dig into the code to try to figure out where the problems are, but what if it wasnt? What if it was easy to see it, what if we had a dashboard that points right to the problem and even told you how bad a problem it was, and what if it told you how often this performance problem occurs during peak times of business. And what if it could help you not only find bottlenecks to speedup your website, but also help with other customer issues. Would it make your company look like a rockstar because you can find and fix problems really really quickly. More speedier then your competitors do?? More cheaply then your competitors can?? This means you have better margins in your PHP business and happier customers at the same time. If you like the potential of improving speed as far as TTR, then take a look at the following diagnostics webinars, it will showcase some of the same technology our Zend experts use here. So start invest in your performance tuning and troubleshooting chops and get great customer satisfaction speed (and loyalty) for your business.

Announcing FOSE2010, #1 Gov focused Tech event of the year

As many of you know from my previous posts about Government use of PHP, I'm a big supporter of how opensource can help in the public sector including one of my 2010 predictions that this could be a great growth area this year. Well if you're like me, and want to learn more about how the government is using technology, there's an upcoming conference called FOSE in March 23-25. FOSE 2010 is the place to discover opportunities and solutions along with changing expectations for government IT professionals.

Register today for the FOSE 2010 experience: http://www.fose.com

You can expect:

- 3 days of IT resources helping you navigate today’s shifting tech landscape

- 2 full conference days packed with education on emerging technologies, trends, and new improvements to existing solutions

- Thousands of products on the FREE* EXPO floor allowing you to gain one-on-one insight into the capabilities of our exhibitors through demos, theater presentations and FREE* Education.

- Visit the Data Center and Virtualization Pavilion, CloudCamp, and the Cloud Computing Pavilion.

*FOSE is a must-attend free show for government, military, and government contractors.

It’s time to register and reserve your place at FOSE http://www.fose.com

Friday, January 8, 2010

Trends in 2010 and beyond

Its another new year and the start of the next decade for the internet and some interesting new stories have hit the net already in 2010. One very popular pastime for many in January in the blogosphere is to look into the crystal ball and predict future trends. And as with most predictions this time of year, some are just a logical progression or evolution of an idea, others are nothing short of guesstimates of the future. But in both cases it is fun to speculate on what will happen in the future and then look back and see how good a fortune teller you can be. If someone told me last week that a terrorist would board a plane and try to set off a bomb in his underwear, I would have called him crazy. But plenty of people predicted that there would be a terrorist attack on a plane this past year, that just seemed statistically probable based on prior attempts, and easy to deduce because its such a general prediction. Since I know that I have no special powers to look into the future, I will stick to logical trends I see that are a natural evolution of ideas for the PHP web in 2010.

THE INTRANET IN YOUR POCKET TREND - is a clear one for everyone to see since we have been in the middle of this trend since the first PDA’s were introduced 17yrs ago, (hint for you are a history buff - Newton was also an Apple product with limited modem connectivity back in the beginning) and now with the next gen of mobile web devices, everyone should be preparing for PHP web applications that will also fit in both form factors, PCs as well as mobile phone /tablet devices. There are far more phones then computers, and the smartphone will eclipse the PC as the dominate device to cruise the web. PHP has been a leader in this mobile space also, as some of the most popular iPhone apps come from some popular PHP sites you probably know very well. If you’ve ever used facebook or anyone of Zynga’s social games (FarmVille, Mafia Wars, Live Poker, etc.) you are already using some of the most popular iphone apps that are written with PHP backends. There are plenty of other examples, the big trend in 2010 mobile will be the explosion of enterprise apps for the iPhone and other tablets/smart phones like Google’s Android. Again most of this driven by smartphone's web browser and accessing corporate web apps built for smaller screen templates. I think PHP systems had the biggest customer growth so far on the iPhone because it was such an agile environment to take Web and Social Networking ideas and implement them quickly. For the enterprise, the mobile enable FOSS application will be helping to bring these same advantages to internal resources and information. Did you know the most popular PHP opensource blogging app Wordpress is on the iPhone already? So is Drupal, it has an iPhone template, so does Magento, SugarCRM, and Moodle, etc. This makes it easy for enterprises to make the web application ready right out of the box with these PHP applications. I predict that there will be many many more free opensource PHP apps with smartphone optimized templates that make intranet resources and content accessible to the mobile workforce. Wonder what SugarCRM would be like if it tracked field workforce via mobile GPS?, UPS is already doing something like this with its field force. This and other mobile enabled enterprise apps will be a key focus for enterprises in 2010. And yes, I know that these smartphones have a zoomable web browser for any website to work, but making it truly optimal for the small form factor to collect statistics from the road, or get reports on the road via touchscreen typing/voice lookups will require tweaking of most intranet app GUIs to be road warrior ready and useful.


MORE OPEN THEN CLOSED BUSINESS TREND – This trend has been getting a lot of press in the mobile web space lately also. As many experts are talking about the Google/Apple wars, and predicting that Google’s more open Andriod will ultimately hurt Apple’s more proprietary iPhone system. Even though I’m an Apple fanboy, I must also agree with this prediction as history has proven in the past the right mix of Open and Proprietary is the key to business success longterm. I was watching Cramer on MSNBC today who tried to tell investors that its not an either/or death match between Apple/Google, but I disagree it will ultimately shake out to one of them owning 60% or more of the smartphone market, and the others with the rest. It happened to Apple ][ that dominated personal computers until the more open IBM PC took over, then again the more closed MAC .vs. more openly available Windows on any PC. As a smart man said once, “it’s déjà vu all over again” and will happen again with smartphones. More open systems tend to compete better in free markets, and is one of the reasons most closed and proprietary firms are opening up and releasing projects based on open source. Even Microsoft has spent a lot more resources on open source initiatives this past year and on windows PHP in particular. Enterprises and Government agencies are also betting big on open initiatives in the new year, leveraging open source projects like Wiki’s, Blogs, and CMS’s inside the firewall. Some of the most popular open applications on the web are PHP apps and in 2010 I hope to see a few new ones and updates of popular ones. If last years batch is any indication of what will be released this new year, it’s going to be very exciting. I also predict new Zend Framework open source apps to be announced in 2010, it has become the most popular PHP RAD framework so I think we are at a tipping point in the community now to see the next generation of free open source apps based on it to come out. Remember the first app that was designed in the beta version of Zend Framework by IBM back in 2007, it was called QEDWIKI and other projects like Magento have since gone public to great success and acclaim. What open source Zend Framework ideas will we see in 2010 I do not know, but I talk to many internet leaders who are releasing great Zend Framework based ideas commercially every week.

BAD ECONOMY in 2010 =MORE IT AUTOMATION TREND - ok so this one is another easy trend to spot, I promises you I’m working up to my best predictions in this article like a countdown they will get more futuristic as you read on. But I do envision a continuation of automation because of the bad economy that will continue in 2010 and will make IT automation projects even more important to the bottom line. PHP is great at exchanging data as a client or a server in SOAP, REST and XMLRPC for automation of system to system communication. Webifying client/server and mini/mainframe apps will also be accelerating this year. Rapid Application Design will be a focus for most firms where PHP excels and Zend Framework /developer solutions (and other PHP frameworks) improve the productivity of your staff and quality of the web projects. Of course I’m biased to Zend’s developer suite as it’s a complete, well integrated and powerful solution for enterprise development, check out the new capabilities we just rolled out at end of 2009 and some beta videos of 2010 advances here for a look at our future versions.


UTILITY COMPUTING TREND – In a past life I worked for Loudcloud that was an early pioneer for utility computing in the data center. This was before vmware and cloud computing like xen, and the benefits to fully managed servers as a utility from a business standpoint was very compelling even back then. It was in its infancy at Loudcloud but many advances since 2000 have made managed virtualization and cloud services easier to implement, more economical and a standard in >90% of the fortune 1000 datacenters. PHP applications and architecture easily scales in this new more dynamic utility compute environment. PHP has a simpler architecture then Java app servers so its easier to implement, and scale and build dynamic web businesses in virtual and cloud environments. This is one reason VMware acquired springsource this past year as it’s a harder problem to manage and solve in Java then it is in PHP. With Zend PHP in the cloud, you will be able to take one of the oldest FOSS projects on sourceforge.org (SquirrelMail developed in 1999) and deploy it on the Amazon EC2 cloud or VMware clusters, can you do that to a Java app written in 1999 without recoding?

We here at Zend also made some new advances to ensure PHP web applications can take advantage of the specialized cloud services without locking the web app from working on only one service, so you could build a PHP app that can work across all the many cloud services and virtual datacenters without having to recode it for each. I have a few customers who have a mixed, inside outside cloud strategy and want to build a single codebase that can be deployed on internal VMware and external cloud services like Azure or EC2, but taking advantage of database service or queueing services are different in all the offerings today. And as more services are released this problem only grows for web enabled cloud computing. Our ZendPHP/partner cloud initiative is called SimpleCloud API and the software will support IBM, Microsoft, Rackspace, Amazon, Nirvanix, VMware and GoGrid services. I predict that Zend will announce new capabilities to help make management and monitoring in a cluster far easier for PHP apps. Ok, so this is more of insider information then forecast but stay tuned for our upcoming Zend Server 5.0 bundled capabilities in this space also.

SITUATIONAL APPS DRIVE GOV OSS ADOPTION TREND – New types of data are become realtime on the web. For example, today in California there was an earthquake, if you Googled it there was already references to people tweeting and blogging about it 2min after the quake. This is impressive, what other realtime data will you be able to search for in the future and what kinds of PHP ideas could be built to data mine the information that now will be available like this. Will someone create a mobile app to measure the public’s reaction to statements made by elected officials so that the message could be tweaked faster and refined. I bet Janet Napolitano could have used such an app after she made her comments that the system was working from the attempted Christmas day bombing. Maybe she would have quickly followed up in the next news briefing with a refined message if she was using such an application. This type of statistical web apps and situation management solutions are become very important to the US government, they are known as "Situational Apps" and there are a lot of PHP portals that could be used to quickly build these types of portals for government employee's to manage and respond to certain critical situations. An example would be an online work environment (think of it as a private portal/social network) used to improve US security procedures used by cross functional US agencies involved in the Christmas day bombing scenario. And the participants are both federal, local and commercial that are working together to fix this problem and what better online resource to keep these traditionally isolated groups working together, then the same technology that works out in the internet, open source PHP apps will be a key part of the solution I believe. If you viewed the QEDWIKI video earlier that's just one (non-gov) example of how PHP can build these easily, it was also the very first "Situational App Generator" that IBM created using Zend Framework and PHP back in 2007, there's been a lot progress since. Mindtouch project is a more current open source PHP based example that I like a lot for some of these also. This trend is still in its infancy in Washington, but I learned a lot about it when I attended GOSCON this past year. SO I predict that this will be a huge huge growth area for PHP business and the killer app for open source PHP adoption by the US government and other public agencies.


So let me end my article with at least one wayout 2010 prediction from the crystal ball. I predict that unlike the shoe bomber incident of last year that caused us all to have to take off our shoes at airport security, this year we will not be told to take off our underwear and place it on the conveyor for a scan. I know this is crazy talk, given how safe things have been since we all started taking our shoes off, but I fear that the underwear bomber scanning will cause more delays and possibly conceive more babies at airports if the TSA were to solve the problem the same old government way. I know, I know I really put myself out there on this last prediction, but we’ll revisit this in December 2010 and see how things shaped up.

If you have any observations you’d like to share about PHP or Zend product feature suggestions for the new year, please post them here, I’ll put some links of good posts I have read so far. Happy New Year

Thursday, December 10, 2009

Holiday Queues

Its that time of year again, Holiday’s are coming fast, and for me and other geeks we get to play with some new gadgets that we couldn’t get to all year. In the office I also use this time of year to try new things and experiment a bit because the last week in December and early January is usually a perfect time to do this for me. There’ve been some major new releases of OSS apps that I am planning to install and work with ontop of Zend Server in 2010. As my regular readers know, in the past I have already written about Zend Server and how it can significantly improve performance and identify problems in some popular OSS apps like Drupal, Joomla, Wordpress, SugarCRM and Magento.

At the holiday break I will take the latest beta released versions of phpBB 3.0x & Magento 1.4x and experiment with Zend Server’s Job Queue. When you work on a PHP project from scratch its easy to implement new capabilities, but with existing code bases like phpBB and Magento there seemed to be some very obvious places to use Job Queue and enhance performance. If you’re not familiar with Queueing, we had a good webinar this past month from Shahar the lead engineer on this. For places in the project with long-running tasks, such as order processing, catalog importing/exporting, content indexing or sending an email just to name a few obvious ones. These tasks can unnecessarily load your web server and increase application response times for the end user because the processes are sequential. Zend Server 5 beta features Job Queue support, enabling you to offload the execution of such tasks to a parallel request, or even to a different server when needed. I’m going to keep it simple and parallelize a lot of long running tasks in these projects. We have a draft of the API online already, you can find the whitepaper here for Job Queue if you want to experiment along with me.

As far as getting ready for the holiday’s, I’m am still having a hard time finding one last gift for my young nephew (who I know will be an aspiring developer/entrepreneur some day), I wanted to get him a collection of stuffed Open Source mascots as inspiration when he gets older. I already have the uber rare Mozilla, ElePHPant, Tux, but am missing and cant find a Mysql Sakila dolphin but I know some folks have a bunch like this guy in the photo, so am willing to trade PHP Elephpants for one or two and complete my stuffed animal LAMP stack. Contact me if you have any spare ones, and have a happy holiday.

Thursday, November 12, 2009

PHP at Government Open Source Conference 2009

I attended the first GOSCON 2009 (Government Open Source CONference) in DC last week and learned a lot about the government’s use of open source software (OSS). There were two major announcements at the conference, first by the DoD and the second by the White House.

The keynote with DoD CIO David Wennergren was the most anticipated event for most at this conference as his talk was about the newly released DoD guidance document that clarifies the use of open source software at the agency (and is also considered a template for all the rest.)

To quickly sum it up for anyone who missed his talk, he reiterated current federal law, which states Government agencies HAVE to look at commercial items [when building out new capabilities] and that open source is considered commercial software also. So a tip to all of us in OSS land, when dealing with the government agencies, if the agency you are working with already knows how to budget and procure commercial software today, then the procedures are the very same for agency projects that want to use open source software.” It’s that simple! then he went into specific examples in the DoD.

A lot of newspaper articles were written about how this will revolutionize the software landscape in Government, but I would argue that the revolution is already underway as most progressive government agencies are already using open source software. Though I would also say that this document will make OSS use more common place now as many in the room seemed to struggle with licensing and procurement in the past (based on the questions being asked at the conference it was just as much about OSS and the services for the project.) This memo clarifies a lot of misunderstandings within DoD (and the government) on how and when open source software can be implemented. This evens the playing field because it says open source solutions should be allowed to compete when the department is implementing a major system.

Did we really need a memo that reiterates what most enterprises have already discovered: open source saves money, is scalable, secure, and easy to adapt? Not for the early government adopters like the White House. The other big presentation at GOSCON was about the Whitehouse.gov switching its site to use open source software PHP and Drupal. For many of you who have been following my blog articles, back in February I had an article about “The most Tech Savvy use of OSS in Government” and that distinction went to BarackObama.com and the folks at Blue State Digital who took their ideas and implemented them in open source PHP to change the state of politics and campaigning forever, read more here. A little over 9months later, the president’s office does it again, and revamps one of the most important government sites, Whitehouse.gov and again proves why the most Tech Savvy president’s office uses open source PHP to make fast, flexible and frugal government a reality.

After months of planning the White House has ditched the proprietary content management system that had been in place by prior administrations in favor of the latest version of the open-source content management system based on PHP. As first reported by the AP The Obama new media team, with a few months of executive branch service under their belts, decided they needed a better development environment for the White House web presence.

After Obama’s new media team entered office and started tweaking the old WhiteHouse.gov, they decided they needed a more flexible development environment for the White House web presence. They wanted to be able to more quickly, easily, and gracefully build out their vision of interactive government. General Dynamics Information Technology (GDIT), the Virginia-based government contractor who had executed the Bush-era White House CMS contract, was tasked by the Obama Administration with finding a more flexible alternative. The ideal new platform would be one where dynamic features like question-and-answer forums, live video streaming, and collaborative tools could work more fluidly together with the site's infrastructure. The solution, says the White House, turned out to be a PHP based content management system. That's something of a victory for Drupal and PHP (not to mention open-source) community.

Is this the first move to a more open and transparent government even in code? As government developers start to embrace OSS programming and other open systems like open - forges, frameworks, languages and applications etc? I think the White House's move to open-source software signals a move towards this idea that collaborative programming can also inspire (or at least support) more open government systems.

As PHP programmers we all have seen how code openness (transparency) has made a huge impact to our own systems. When some other group/programmer wants to add or improve something to a shared OSS system, they can with open systems or they can just fork your work as a starting point. When we PHP programmers need a new system and there was already an open source project that can help get us started quickly, we just went to sourceforge.org and downloaded the PHP application to get started (just like the Whitehouse.gov site team did.) If we needed to implement something truly innovative and revolutionary, we could still use PHP to build it, even hire consultants to do it for us, and yet still we are free to build more into our system at a later date since the code is open to change (just like BarakObama.com online social politics and networking did.)

David Wheeler from DoD made a real good analogy to one attendee who was having a hard time with the concept of open code in government, he said it this plainly after several prior attempts - by comparing OSS project to buying a car for government use, "Do you want your new government car to have a hood that is welded shut because that's what proprietary software is like, or would you rather have a car that lets you open the hood, and lets you pick whatever shop or mechanic you need to improve it, not just today but into the future, and that's what open source using open languages are like" Yup that's what PHP will offer government applications, it will allow them to build IT projects with a hood that can be opened by any programmer (of course with the right approval and security - there is after all a key needed to open that hood, just like the car.)

What I think that these examples start to show us is that transparency in open code can offer transparency and agility when government polices change over time, and that’s real exciting and a huge step forward for Government. Fewer projects will be started from scratch, and more reuse and evolution of ideas will undoubtedly become the norm with more OSS proliferating government. OSS use also allows the government to reuse ideas and code that has already been proven in commercial settings. If you look at all the opportunities for different government agencies to not only reuse much of OSS code that’s out on the web, but also share more unique and US government focused projects on its own internal forges. (Forge.mil is another agency example of the forward thinking DoD open source initiatives) Many of the large enterprises that I work with have this very same type of open source policy for internal business as well as leveraging the OSS forges and applications. I’m sure this will inspire other governmental agencies (and contractors) to think about how best to leverage and share OSS code to make fast, flexible and frugal government a reality in the US. So let me know how you’re using PHP in government today and making fast, flexible and frugal government possible.

Other articles:



Monday, October 26, 2009

Zend Server 5.0 Intro to Code Tracing

Zendcon2009 ended this past week, and we introduced a new beta version of Zend Server 5.0 with several new features. In this article and video blog I will review Code Tracer. Think of it as a blackbox flight recorder for your PHP applications. When something goes wrong with an airplane, everything is saved during the flight onto the blackbox to help the investigators figure out what went wrong. What if you can do that in production with your PHP applications to figure out problems when they happen, now you can with the new Zend Server 5.0 Code Tracer.

In this video I will setup Zend Server Tracer to capture a problem from an open source application called phpExcelReader . There are two problems with this project once you download it from sourceforge that Zend Server can help you quickly identify it and fix it. I wanted to keep this first example real simple to show how to setup the CODE TRACE and used a real world FOSS application with the typical white screen problem that is so common with development. There are far more difficult problems that this feature can help you solve, including performance issues, code problems and even segfault issues when they happen a trace is the only thing that can help you figure it out. Let me know what problems this feature has helped you identify in your project, drop me a post here.

If you want to follow along be sure to get the new beta software from here:
and review the whitepaper for TRACING:


And to register for the free webinar that will go into more details and a longer demo of Code Tracer, click here.