How to Get Out from Under an Obsolete Legacy Technology by
Duncan KenzieNo, no, no. We're not talking about abandoning the System i. We're talking about taking full advantage of it.
Published April 2007
This article is about getting out from under an obsolete legacy technology. I suspect many of you are thinking the same thing: "Oh, no. Here we go again—another article whose thrust is to move us off the iSeries platform." Well, you'll be happy to know you can keep reading, As a diehard System i fan, I'm rankled by the term "obsolete legacy." I believe the iSeries platform is cutting edge and positioned for many future years of life because it's reliable, scaleable, and secure.The "obsolete legacy technology" of this article is a reference to restrictive, proprietary, dead-end development languages and infrastructure, not the System i hardware and operating system platform. Although its detractors and competitors like to refer to the System i as "legacy," to me it's anything but that because of IBM's commitment to supporting all necessary TCP/IP protocols required to run a modern suite of business applications. And that's the real thrust of this article: If you haven't already done so, you need to move your core business applications to the Web, and you should be using the System i as your primary Web server. If you don't, your competition is going to get the better of you. Whether you are a software vendor or a business in a vertical industry with green-screen applications, you will be competing with other iSeries or Wintel-based IT shops that are Web-based. These competitors can deliver more accessible and meaningful information at less cost than green-screen. If you are still running primarily green-screen five years from now, your business will likely suffer greatly. The technologies I consider to be obsolete are primarily the traditional programming language and develop environments of the iSeries, along with the green-screen 5250 paradigm. Let's examine these in more detail and look at some of the best alternatives to them. I'll also explain what sort of skills your staff will need so you can take maximum advantage of them as soon as possible.
Languages
On the System i, the languages that have traditionally been used most widely are RPG and COBOL. These were created in the late '50s and '60s and have been used extensively on IBM's midrange platform for the last 30 or so years. RPG is the dominant language, for both in-house applications and older commercial offerings, so I'll focus on its limitations for modern development. RPG is a business language, meaning it has widespread support for data types used in simple business computations such as pricing routines and totaling for reports. It also has extensive integrated database support. It has evolved to include facilities for interactive programming to make it a relatively productive language for writing green-screen applications. For example, subfile support, which was introduced in 1979 with the System/38, made it relatively easy to write list-style programs with paging routines for scrolling through large lists of data. For modern applications requiring extensive Web functionality, though, it has many weaknesses:
RPG is procedural. Contrary to what you'll find on Wikipedia about RPG, for example, it is not an objected-oriented language, having no standard OOP features such as methods, properties, inheritance, or polymorphism. It also lacks advanced features—such as garbage collection—that are available in Java, JavaScript, C#, and other object-oriented language environments.
RPG has a proprietary syntax, especially for data description code and output code. While free-format RPG has moved RPG more to the mainstream, it still has many quirky constructs when compared to languages such as C, C++, Java, or even Visual Basic. This makes it difficult to convince any new-hires fresh out of college to see the value in learning this language. It's most likely not anything like what they've seen during their education, and many grads will view working with RPG as a step backward, given its lack of true OOP features and its proprietary nature.
The coding environments for RPG are primitive, expensive, or unproductive. SEU, the native tool for editing RPG, is text-based and severely lacking in productivity features that are standard in Windows development environments such as Microsoft Visual Studio. True, there is WDSc, but this is difficult to learn, requires large amounts of memory, and is slow at communicating with the iSeries when downloading source files. And the recent announcements from IBM haven't helped things, with features previously available in CODE/400 now only available at great cost per developer.
RPG has no native Web programming support. This is probably its biggest weakness. While it was good at 5250 green-screen programming, you must either write a ton of code to call Web APIs, construct your own Web functionality, or acquire a third-party product to help you. Fortunately, there are several good third-party offerings available to help address this deficiency, some of which are some form of open-source while others are available for a license fee. I'll get back to that later on.
Getting out from Under RPG's Baggage
The dilemma many IT managers face is the problem of the mountain of legacy code written in RPG. How can they move the business functionality to the Web? Several options are available: Webfacing This involves screen-scraping the 5250 screens and rendering them as Web pages according to predetermined rules. Several tools from IBM and third-party vendors can help with this. Each one has varying degrees of customization you can apply to the output. The main argument in favor of Webfacing is that you don't have to take an all-or-nothing approach. Your application will, in many cases, just run straight out of the box. Here are the reasons not to go down this road:
Many of the Webfacing tools require an interactive session to run in the background, thus using interactive tax, which makes hardware costs much more expensive. One of IBM's solutions and at least one third-party vendor solution now offer Webfacing that doesn't require interactive sessions, but this is only for V5R4 and higher.
Webfacing is not pretty unless you invest time and effort in customizing screens. You should consider investing this time in true modernization instead, which involves keeping back-end business and database logic intact while using a true Web protocol for the user interface. With the right tool, you may find that the additional amount of work (versus Webfacing) is negligible.
Webfacing is itself often just another short-term solution. You are dependent on the Webfacing software and a unique skill set for your staff that can't really be leveraged for use in true Web programming.
Migrating Existing Code to New Development Environments This approach involves using some kind of code conversion tool to take existing RPG code and move it into a Web development environment. A couple of "reverse engineering" products use this philosophy, as well as updated tools from the '80s and '90s that allow you to migrate your code into a data model and reuse it. Here are some of the problems with this approach:
The products often have long and steep learning curves.
They often require a complete commitment; you can't modernize easily in stages.
They are expensive to license.
They are based on technology that originally came from the green-screen world, so vestiges of their heritage impede their usability.
Again, your staff is learning a skill set that is relatively unique to the products and not easily transferable to more widespread development approaches.
Rewriting Applications in Java I mention this only because this approach has been touted so much by IBM and the iSeries press in the last six years. I think it's fair to say this approach has proved to be a failure for the majority of iSeries shops, except those companies with large and highly formalized development groups and big servers. These have typically been the top-tier Fortune 100 companies, but for small and medium-sized businesses (or small departments within large businesses) this approach is completely unsuitable. Here's why:
Java is not easy for iSeries developers to learn. The language itself maybe fairly straightforward, but the infrastructure—libraries, custom objects, etc.—requires a lot of learning. Being proficient in Java programming has, over the years, involved dozens of continually evolving sub-skills and knowledge bases—from J2EE to JSP to ANT to Struts to EJB to JMX to JMS to JNDI to AWT to Swing to JDO to JAAS (and the list goes on and on). You may or may not need any one or more of these, but you certainly can't rely on "just Java." Then there are JAR files and WAR files and RAR files and CLASSPATHs and more.... Some of these involve standards that change over time, so you may have to go back and relearn and re-implement and update various portions of your application, in essence "running to stand still."
Developers' experiences with the major Java development environment, WDSc, have been mixed. They find it difficult to work with and cumbersome to install. Also, many developers just don't have PCs powerful enough (especially laptops) to use it, although IBM is attempting to address this issue with new, "lighter" versions of WDSc.
RPG and Java are very different languages, requiring completely different programming skill sets. Java requires object-oriented programming techniques, which RPG doesn't even support. RPG uses much simpler math constructs than Java. You often have to do convoluted things with Java to perform calculations that are extremely simple in RPG.
WebSphere is required to run Java Web apps on the iSeries. WebSphere is notoriously slow in many implementations and complicated to install and configure.
Integrating New Functionality with Existing Processes This approach requires a lot of effort up front on your part, but it has potentially big dividends in payoff. It involves an in-depth analysis of your legacy code base in order to determine how user-interface code can be separated from database logic and business rules. In order to do this, you need a tool that can support calling existing RPG routines, whether they are in programs, modules, or service programs. You should also look for one with a highly productive Web development environment, one that supports all the programming constructs required for true Web programming: HTML, CSS, JavaScript, SOA, and now AJAX too. Here are some of the steps to take:
Get familiar with true Web applications. Run some commercial Web apps on their sites. For example,
Sugar CRM is an open-source CRM project with online demos. You can run the application to see how well-designed Web apps work. Start thinking about the kinds of UI features you'd want for your apps. For example, it's nice to have pop-up calendars for date fields, instead of forcing your users to have to type a date in a correct format. Drop-down boxes for small lists of values often increase users' productivity (as opposed to green-screen interfaces), and for developers, they eliminate the need for code to validate and issue error messages.
Take an inventory of your programs and categorize them by degree of complexity. Consider completely rewriting your file maintenance programs, for example; that will often be more efficient than trying to convert them and will give you a true Web look and feel.
Separate back-end routines from green-screen interface code. Write well-defined modules from existing code with clear public interfaces. For example, a pricing routine may be wrapped inside a module that has a list of expected parameters and then returns a single value to the calling piece of code.
Prototype your Web pages with HTML layouts. Don't worry about the back-end code actually doing anything meaningful; just get the design of each page established. You'll find, in the process of doing this, a lot of common design features emerging that you can leverage by sheer repetition—for example, placement of Submit, Next Page, Previous Page, and Cancel buttons or the way links to supplementary pages (such as searches and lookups) will appear. Spending the time up front to do this groundwork will pay off many times over when it comes time to write the actual code.
Consider using a tool from IBM or a third-party vendor to help you with page design. Besides WDSc, several good iSeries Web application development tools that don't require the "all or nothing" approach can help you.
This last approach leverages your existing RPG for the back-end code. Despite the weaknesses of RPG for Web programming, it's still appropriate to use for database and business logic. However, what technologies should you use for the Web component? Some of the major options are Java, CGI programming, or PHP. I've already dealt with why you should avoid Java. CGI programming on the iSeries has great advantages if you have a good development tool to write it in. CGIDEV2 is a CGI-based set of functions that is available for free, but the old adage "you get what you pay for" definitely applies here. These have limited functionality, have poor optimization for performance, and require manual coding in RPG. Many people use them as an introduction to learning some of the programming paradigms for Web programming, for which they are useful. But for long-term usage, you should look at the various third-party iSeries Web application development tools. PHP is the new darling of the iSeries world. It's a Web development language that started as open-source but is now commercially supported by Zend. IBM and Zend have an agreement to distribute the Zend core for free with the operating system. The Zend core is an extension to the Apache Web server that provides functionality similar to that of the CGI extension. You can also compare it to the WebSphere Web Application Server conceptually; it provides the services that enable you to write your Web applications, much like system APIs at the operating system level do for traditional green-screen and batch programming. Here's why you should seriously look at PHP:
PHP functionality on the System i is a no-charge item.
It's been around a long time (in Internet time) and is proven. For example, much of Yahoo!'s site is powered by PHP programs.
It's available on multiple platforms, including Linux, UNIX, FreeBSD, and Windows.
It's a simple and productive language to learn—so much so that some of its detractors refer to it as a language for "script kiddies." But this simplicity means your development staff will become successful with it quickly.
Despite its simplicity, it is incredibly feature- and function-rich. The list of standard PHP functions runs into the hundreds.
It's easy to set up and configure.
It runs fast on the iSeries. Our company has run internal tests comparing the performance of a PHP program to one of our optimized CGI programs and found no difference.
It can easily call existing RPG programs. Again, our performance tests showed no difference between a CGI RPG program calling another RPG program and a PHP script calling the same RPG program.
It can use multiple databases, including DB2 on the iSeries and MySQL. MySQL is an open-source competitor to UDB2 that has widespread acceptance in the Web programming world. The support for MySQL gives you flexibility in that you can create a PHP application to run using the native database on the iSeries and then port it to other platforms running MySQL by changing only the database coding.
There are lots of good development environments for working with PHP, including Zend Studio and many shareware or low-cost editors.
PHP has IBM's full support. IBM has worked very closely with Zend to make PHP a technical success.
One caveat: Your operating system will need to be at V5R3 or V5R4 to run PHP, so if you are not there yet, I encourage you to upgrade.
There's No Silver Bullet
All the approaches we've looked at have pros and cons. Every one requires some degree of strategic planning, effort, and education for you and your team of developers. If you leverage your existing RPG code assets for back-end business and database logic and integrate that with true Web programming techniques and environments such as CGI or PHP, in a relatively short amount of time, you can successfully produce true Web applications that you and your users will be happy with for years.
Duncan Kenzie is President and CTO of
BCD Technical Support, the development and support group for
WebSmart, a popular iSeries Web development tool, and
Nexus, a portal product specifically designed for System i, iSeries, and AS/400 servers. Duncan has 29 years of experience on the midrange systems platform creating software for both green-screen and native Web environments.
How to Get Out from Under an Obsolete Legacy Technology by
Duncan KenzieNo, no, no. We're not talking about abandoning the System i. We're talking about taking full advantage of it.
Published April 2007
This article is about getting out from under an obsolete legacy technology. I suspect many of you are thinking the same thing: "Oh, no. Here we go again—another article whose thrust is to move us off the iSeries platform." Well, you'll be happy to know you can keep reading, As a diehard System i fan, I'm rankled by the term "obsolete legacy." I believe the iSeries platform is cutting edge and positioned for many future years of life because it's reliable, scaleable, and secure.The "obsolete legacy technology" of this article is a reference to restrictive, proprietary, dead-end development languages and infrastructure, not the System i hardware and operating system platform. Although its detractors and competitors like to refer to the System i as "legacy," to me it's anything but that because of IBM's commitment to supporting all necessary TCP/IP protocols required to run a modern suite of business applications. And that's the real thrust of this article: If you haven't already done so, you need to move your core business applications to the Web, and you should be using the System i as your primary Web server. If you don't, your competition is going to get the better of you. Whether you are a software vendor or a business in a vertical industry with green-screen applications, you will be competing with other iSeries or Wintel-based IT shops that are Web-based. These competitors can deliver more accessible and meaningful information at less cost than green-screen. If you are still running primarily green-screen five years from now, your business will likely suffer greatly. The technologies I consider to be obsolete are primarily the traditional programming language and develop environments of the iSeries, along with the green-screen 5250 paradigm. Let's examine these in more detail and look at some of the best alternatives to them. I'll also explain what sort of skills your staff will need so you can take maximum advantage of them as soon as possible.
Languages
On the System i, the languages that have traditionally been used most widely are RPG and COBOL. These were created in the late '50s and '60s and have been used extensively on IBM's midrange platform for the last 30 or so years. RPG is the dominant language, for both in-house applications and older commercial offerings, so I'll focus on its limitations for modern development. RPG is a business language, meaning it has widespread support for data types used in simple business computations such as pricing routines and totaling for reports. It also has extensive integrated database support. It has evolved to include facilities for interactive programming to make it a relatively productive language for writing green-screen applications. For example, subfile support, which was introduced in 1979 with the System/38, made it relatively easy to write list-style programs with paging routines for scrolling through large lists of data. For modern applications requiring extensive Web functionality, though, it has many weaknesses:
RPG is procedural. Contrary to what you'll find on Wikipedia about RPG, for example, it is not an objected-oriented language, having no standard OOP features such as methods, properties, inheritance, or polymorphism. It also lacks advanced features—such as garbage collection—that are available in Java, JavaScript, C#, and other object-oriented language environments.
RPG has a proprietary syntax, especially for data description code and output code. While free-format RPG has moved RPG more to the mainstream, it still has many quirky constructs when compared to languages such as C, C++, Java, or even Visual Basic. This makes it difficult to convince any new-hires fresh out of college to see the value in learning this language. It's most likely not anything like what they've seen during their education, and many grads will view working with RPG as a step backward, given its lack of true OOP features and its proprietary nature.
The coding environments for RPG are primitive, expensive, or unproductive. SEU, the native tool for editing RPG, is text-based and severely lacking in productivity features that are standard in Windows development environments such as Microsoft Visual Studio. True, there is WDSc, but this is difficult to learn, requires large amounts of memory, and is slow at communicating with the iSeries when downloading source files. And the recent announcements from IBM haven't helped things, with features previously available in CODE/400 now only available at great cost per developer.
RPG has no native Web programming support. This is probably its biggest weakness. While it was good at 5250 green-screen programming, you must either write a ton of code to call Web APIs, construct your own Web functionality, or acquire a third-party product to help you. Fortunately, there are several good third-party offerings available to help address this deficiency, some of which are some form of open-source while others are available for a license fee. I'll get back to that later on.
Getting out from Under RPG's Baggage
The dilemma many IT managers face is the problem of the mountain of legacy code written in RPG. How can they move the business functionality to the Web? Several options are available: Webfacing This involves screen-scraping the 5250 screens and rendering them as Web pages according to predetermined rules. Several tools from IBM and third-party vendors can help with this. Each one has varying degrees of customization you can apply to the output. The main argument in favor of Webfacing is that you don't have to take an all-or-nothing approach. Your application will, in many cases, just run straight out of the box. Here are the reasons not to go down this road:
Many of the Webfacing tools require an interactive session to run in the background, thus using interactive tax, which makes hardware costs much more expensive. One of IBM's solutions and at least one third-party vendor solution now offer Webfacing that doesn't require interactive sessions, but this is only for V5R4 and higher.
Webfacing is not pretty unless you invest time and effort in customizing screens. You should consider investing this time in true modernization instead, which involves keeping back-end business and database logic intact while using a true Web protocol for the user interface. With the right tool, you may find that the additional amount of work (versus Webfacing) is negligible.
Webfacing is itself often just another short-term solution. You are dependent on the Webfacing software and a unique skill set for your staff that can't really be leveraged for use in true Web programming.
Migrating Existing Code to New Development Environments This approach involves using some kind of code conversion tool to take existing RPG code and move it into a Web development environment. A couple of "reverse engineering" products use this philosophy, as well as updated tools from the '80s and '90s that allow you to migrate your code into a data model and reuse it. Here are some of the problems with this approach:
The products often have long and steep learning curves.
They often require a complete commitment; you can't modernize easily in stages.
They are expensive to license.
They are based on technology that originally came from the green-screen world, so vestiges of their heritage impede their usability.
Again, your staff is learning a skill set that is relatively unique to the products and not easily transferable to more widespread development approaches.
Rewriting Applications in Java I mention this only because this approach has been touted so much by IBM and the iSeries press in the last six years. I think it's fair to say this approach has proved to be a failure for the majority of iSeries shops, except those companies with large and highly formalized development groups and big servers. These have typically been the top-tier Fortune 100 companies, but for small and medium-sized businesses (or small departments within large businesses) this approach is completely unsuitable. Here's why:
Java is not easy for iSeries developers to learn. The language itself maybe fairly straightforward, but the infrastructure—libraries, custom objects, etc.—requires a lot of learning. Being proficient in Java programming has, over the years, involved dozens of continually evolving sub-skills and knowledge bases—from J2EE to JSP to ANT to Struts to EJB to JMX to JMS to JNDI to AWT to Swing to JDO to JAAS (and the list goes on and on). You may or may not need any one or more of these, but you certainly can't rely on "just Java." Then there are JAR files and WAR files and RAR files and CLASSPATHs and more.... Some of these involve standards that change over time, so you may have to go back and relearn and re-implement and update various portions of your application, in essence "running to stand still."
Developers' experiences with the major Java development environment, WDSc, have been mixed. They find it difficult to work with and cumbersome to install. Also, many developers just don't have PCs powerful enough (especially laptops) to use it, although IBM is attempting to address this issue with new, "lighter" versions of WDSc.
RPG and Java are very different languages, requiring completely different programming skill sets. Java requires object-oriented programming techniques, which RPG doesn't even support. RPG uses much simpler math constructs than Java. You often have to do convoluted things with Java to perform calculations that are extremely simple in RPG.
WebSphere is required to run Java Web apps on the iSeries. WebSphere is notoriously slow in many implementations and complicated to install and configure.
Integrating New Functionality with Existing Processes This approach requires a lot of effort up front on your part, but it has potentially big dividends in payoff. It involves an in-depth analysis of your legacy code base in order to determine how user-interface code can be separated from database logic and business rules. In order to do this, you need a tool that can support calling existing RPG routines, whether they are in programs, modules, or service programs. You should also look for one with a highly productive Web development environment, one that supports all the programming constructs required for true Web programming: HTML, CSS, JavaScript, SOA, and now AJAX too. Here are some of the steps to take:
Get familiar with true Web applications. Run some commercial Web apps on their sites. For example,
Sugar CRM is an open-source CRM project with online demos. You can run the application to see how well-designed Web apps work. Start thinking about the kinds of UI features you'd want for your apps. For example, it's nice to have pop-up calendars for date fields, instead of forcing your users to have to type a date in a correct format. Drop-down boxes for small lists of values often increase users' productivity (as opposed to green-screen interfaces), and for developers, they eliminate the need for code to validate and issue error messages.
Take an inventory of your programs and categorize them by degree of complexity. Consider completely rewriting your file maintenance programs, for example; that will often be more efficient than trying to convert them and will give you a true Web look and feel.
Separate back-end routines from green-screen interface code. Write well-defined modules from existing code with clear public interfaces. For example, a pricing routine may be wrapped inside a module that has a list of expected parameters and then returns a single value to the calling piece of code.
Prototype your Web pages with HTML layouts. Don't worry about the back-end code actually doing anything meaningful; just get the design of each page established. You'll find, in the process of doing this, a lot of common design features emerging that you can leverage by sheer repetition—for example, placement of Submit, Next Page, Previous Page, and Cancel buttons or the way links to supplementary pages (such as searches and lookups) will appear. Spending the time up front to do this groundwork will pay off many times over when it comes time to write the actual code.
Consider using a tool from IBM or a third-party vendor to help you with page design. Besides WDSc, several good iSeries Web application development tools that don't require the "all or nothing" approach can help you.
This last approach leverages your existing RPG for the back-end code. Despite the weaknesses of RPG for Web programming, it's still appropriate to use for database and business logic. However, what technologies should you use for the Web component? Some of the major options are Java, CGI programming, or PHP. I've already dealt with why you should avoid Java. CGI programming on the iSeries has great advantages if you have a good development tool to write it in. CGIDEV2 is a CGI-based set of functions that is available for free, but the old adage "you get what you pay for" definitely applies here. These have limited functionality, have poor optimization for performance, and require manual coding in RPG. Many people use them as an introduction to learning some of the programming paradigms for Web programming, for which they are useful. But for long-term usage, you should look at the various third-party iSeries Web application development tools. PHP is the new darling of the iSeries world. It's a Web development language that started as open-source but is now commercially supported by Zend. IBM and Zend have an agreement to distribute the Zend core for free with the operating system. The Zend core is an extension to the Apache Web server that provides functionality similar to that of the CGI extension. You can also compare it to the WebSphere Web Application Server conceptually; it provides the services that enable you to write your Web applications, much like system APIs at the operating system level do for traditional green-screen and batch programming. Here's why you should seriously look at PHP:
PHP functionality on the System i is a no-charge item.
It's been around a long time (in Internet time) and is proven. For example, much of Yahoo!'s site is powered by PHP programs.
It's available on multiple platforms, including Linux, UNIX, FreeBSD, and Windows.
It's a simple and productive language to learn—so much so that some of its detractors refer to it as a language for "script kiddies." But this simplicity means your development staff will become successful with it quickly.
Despite its simplicity, it is incredibly feature- and function-rich. The list of standard PHP functions runs into the hundreds.
It's easy to set up and configure.
It runs fast on the iSeries. Our company has run internal tests comparing the performance of a PHP program to one of our optimized CGI programs and found no difference.
It can easily call existing RPG programs. Again, our performance tests showed no difference between a CGI RPG program calling another RPG program and a PHP script calling the same RPG program.
It can use multiple databases, including DB2 on the iSeries and MySQL. MySQL is an open-source competitor to UDB2 that has widespread acceptance in the Web programming world. The support for MySQL gives you flexibility in that you can create a PHP application to run using the native database on the iSeries and then port it to other platforms running MySQL by changing only the database coding.
There are lots of good development environments for working with PHP, including Zend Studio and many shareware or low-cost editors.
PHP has IBM's full support. IBM has worked very closely with Zend to make PHP a technical success.
One caveat: Your operating system will need to be at V5R3 or V5R4 to run PHP, so if you are not there yet, I encourage you to upgrade.
There's No Silver Bullet
All the approaches we've looked at have pros and cons. Every one requires some degree of strategic planning, effort, and education for you and your team of developers. If you leverage your existing RPG code assets for back-end business and database logic and integrate that with true Web programming techniques and environments such as CGI or PHP, in a relatively short amount of time, you can successfully produce true Web applications that you and your users will be happy with for years.
Duncan Kenzie is President and CTO of
BCD Technical Support, the development and support group for
WebSmart, a popular iSeries Web development tool, and
Nexus, a portal product specifically designed for System i, iSeries, and AS/400 servers. Duncan has 29 years of experience on the midrange systems platform creating software for both green-screen and native Web environments.
