(nick-mudge ...) Fearless Programming

I recently attended the first Ignition Community Conference. It was pretty great.

Here's a couple posts about it from the IA forum:

User Conference

Predictions for Ignition's Future

I recently had a problem with a web page created using the Wicket 6 web framework. A button that uploads a file didn't work. When pressed the button didn't do anything.

I investigated the problem and found that the button was using an AJAX function from wicket-ajax-jquery.js. But it wasn't actually AJAX. The call used a hidden iframe to send the multipart form data of the file being uploaded.

Apparently regular AJAX calls cannot be used with HTML 4 to send multipart form data which is required when uploading files. So wicket-ajax-jquery.js simulates AJAX by using a hidden iframe to submit a form.

The problem is that the src attribute on the hidden iframe is set to "about:blank". This right here causes the before mentioned button to not work when IE8's Internet zone security level is set to High. Internet Explorer may give a popup explaining that "about:blank" needs to be added to the list of Trusted sites, or it might not. The indication of what the problem is isn't necessarily clear. In testing on two computers the popup occurs the first time it happens but not after that. On another computer the popup doesn't occur at all.

I don't know if this is a problem with other versions of IE or not.

The solutions I have for this are for users to change their Internet zone security setting to lower than High, or add "about:blank" to their list of Trusted sites, or for the developer making the webpage to use a regular form submission instead of using a simulated AJAX call that uses an iframe.

I wrote this blog post as an experiment because I was surprised that I could not find this problem at all by searching for it with google. Either other people don't have this problem or people do have this problem but it hasn't been written about. I'm curious if much people will find this blog post now that I've written about it.

For the last three years I worked at Calmetrics, the sister company of Inductive Automation. Inductive Automation develops the product Ignition, provides training on it, technical support for it, and sells it. At Calmetrics we built custom software for customers using Ignition.

Ignition is a tool written in Java that makes it easy and fast to build applications. What kind of applications? The Inductive Automation website says it is software for creating Supervisory Control and Data Acquisition (SCADA) systems. However I think it is more general purpose. It is certainly great for and geared for SCADA systems, but all kinds of applications can be built using Ignition.

If you are not familiar with SCADA, here's a definition from Wikipedia:

Computer systems that monitor and control industrial, infrastructure, or facility-based processes.

In my three years I used Ignition to implement or improve inventory systems, plant data logging systems, process tracking systems, plant monitoring systems, Customer Relationship Management (CRM) systems and Human Machine Interface (HMI) systems. HMIs are GUIs that are used to control machinery on the plant floor.

I think a lot of web applications could instead be Ignition applications. Project management, time tracking, email, CRMs, ERP -- data rich applications that you want users to log in to.

A huge benefit to web applications is that they employ a client-server model. There's one server that deploys an application to many clients, the web browsers. This allows changes to the software to be very easy. Just change the code on the server and the clients are automatically updated. And since a web server deploys its application over a network the users can access the application from many physical locations. Of course the Internet enables users to access applications from many places in the world.

Ignition employs this same model. But instead of deploying a client over the network to a web browser Ignition deploys a client over the network to a Java Runtime Environment (JRE) using Java Web Start.

Here's a great description of Java Web Start from www.java.com:

The Java Web Start software allows you to download and run Java applications from the web. The Java Web Start software:
  • Provides an easy, one-click activation of applications
  • Guarantees that you are always running the latest version of the application
  • Eliminates complicated installation or upgrade procedures

So a user goes to an Ignition webpage and clicks on a launch button which causes Java Web Start to download a Java application from the Ignition server. When the application is downloaded the application starts automatically and the user is in business. The Java application running on the user's desktop will communicate to the Ignition server on the network as needed to get data from databases, to update itself when changes are made on the server, etc. After developing applications with Ignition for the last three years I can tell you that this works very well.

The control and interactivity in web applications is getting better and better but it still isn't as good as the control and interactivity in desktop applications. When developing with Ignition you are making a desktop application but you get network benefits of web applications.

Another big benefit that web applications have is that web browsers are ubiquitous on computers and devices and different operating systems. I don't think Java is as ubiquitous but it is still pretty common. Ignition includes support for running Ignition applications on cell phones and tablets. The Ignition server and applications run on Windows and Linux well. For the last couple of years I did all of my Ignition development on Linux, using Ubuntu. That made me happy.

For a long time a problem in web development has been browser consistency, and it still is a problem particularly with Internet Explorer 8 and below not supporting much of CSS3 and HTML5. When developing Ignition applications I didn't have to worry about what might be supported or not. I just focused on developing my applications. If users are using Sun/Oracle's Java 6 or 7, which the large majority of people are, then everything is supported. I have used OpenJDK before with Ignition and it has worked but the Sun/Oracle JRE worked better.

The Designer: the Ignition development environment

Ignition includes in it a web-launchable application called the Designer. This is the development environment used to create Ignition applications. It is launched just like a client Ignition application, from the built-in website that comes with Ignition.

In Ignition applications are called projects. Once a user creates a project in the Designer a user can easily create new windows and access pallets of available GUI components that can be dragged and dropped into windows. Database queries can be written for components and components can easily be connected together so data flows through them.

The drag and drop nature of Ignition, the array of built-in components and functionality and the configuration windows for these things take a lot of programming out of building Java applications. If that disappoints you or makes you think Ignition is limited, then read on. If that excites you, read on.

Many, if not all of Ignition's GUI built-in components were built using Swing, Java's primary GUI widget toolkit.

Ignition provides the usual standard components needed to make GUIs like windows and labels and buttons. It also includes many kinds of components that are used to display data from a database like tables and charts and the dropdown component. It also includes components that are used to display real-time data from Programmable Logic Controllers (PLCs) or a database or other source, such as the Thermometer, the LED Display, the Level Indicator, the Cylindrical Tank and others. It includes components for controlling equipment on the plant floor. It includes other interesting components like the IP Camera Viewer, Barcode component, Paintable Canvas, PDF Viewer and others. Ignition also includes an image component that allows users to add images to their application. Ignition also includes a library of graphics that users can use. A list of components can be found in the Ignition Manual in Appendix A.

Once a user has components on a window they need to be made to do something. They can be made to do a lot of things. A user can use a configuration window to directly connect a property of one component to the property of another component. Ignition has a simple expression language that can be used to connect the properties of components together but also manipulate the data at the same time through a set of available functions. But the real power and flexibility of the system comes from what can be done through scripting with Jython.

Jython Programming

One of my favorite things about Ignition is that it provides its users with Jython 2.5 for event handling and server-side scripting. Jython is an implementation of Python that runs on the JVM.

Ignition provides many built-in tools and windows to configure things. Many users could probably get away with little scripting because Ignition is setup to help users build applications without much programming if they don't want to program. But the configuration windows and built-in functionality limit what can be done and how they can be done. In my opinion Jython scripting (and Ignition's module system) are really what makes Ignition flexible and powerful. When you need more than what is built in, Jython is there to give you flexibility.

So Swing events like mouseClicked, actionPerformed, propertyChange and others are handled by Jython scripting in the Designer. Ignition provides some simple configuration windows for implementing events. These generate equivalent Jython scripts which are then used by Ignition. These configuration windows are useful to learn how scripting is done in Ignition. You can skip the configuration windows and write your event handlers in Jython directly, which is what I do.

Ignition includes a Script Module Editor which allows users to write their own libraries of Jython functions which can be used in event handlers. This is where to put reusable code.

Jython in Ignition is good for people just learning to program, it is good for intermediate programmers, it is good for advanced programmers. It is simple but it doesn't constrain or limit. You can dig into the Ignition software and grab the Swing objects and change them or add new functionality to them. You can do anything that you can do in Jython, Swing and Java. It might not be recommended to mess with the underlying software too much, but you can do it. I once had a customer that needed some functionality on a table component that it didn't have. There was no way to add the functionality other than writing Jython code that dug into the Ignition objects, pulled out the underlying Swing object and added a new event listener that I wrote. Then it worked like a charm. I enjoyed that and it was awesome.

I wrote a lot of Jython code for all kinds of things: manipulating bits of data, processing lots of data, input form handling, database queries, security, safety, coordination of components on windows, navigation, etc. etc.

Ignition includes a built-in library of Jython functions for doing common things like database queries, reading files, navigation, GUI popups, exporting data to CSV or Excel, email, printing etc.

Ignition also provides Jython for server-side programming which runs on the Ignition server instead of in client applications. This is very useful.

Database Support

Ignition has a lot of built-in functionality and support for using SQL databases such as MySQL, PostgreSQL, SQL Server and Oracle. Database queries can be connected to components like tables and graphs to display data. Databases can be accesses and manipulated in Jython scripting easily. Ignition has functionality to easily log data to databases. Ignition has a built-in database browser which is pretty useful. It does a lot of other things with databases.

Ignition Modules

Probably my two favorite things about Ignition are Jython and being able to write new modules for Ignition.

Ignition is made up of a number of separate modules that work together. For example the Vision Module is what provides Ignition with its GUI capabilities. The SQL Bridge Module provides database capabilities. The Reporting Module provides the ability to create reports.

The cool thing is that you can write your own modules in Java, adding your own capabilities, GUI components, configuration, whole systems, and whatever to the Ignition platform.

Inductive Automation provides a ModuleSDK for developing Ignition modules.

I wrote an Ignition module that integrates an Ignition application with QuickBooks. The module includes a SOAP server that is used to communicate with the QuickBooks Web Connector.

I wrote another Ignition module that allows users to select many components in a window and change the values of their properties in bulk. I also added the capability to add event scripts to many components at the same time. This way users could make large changes to their applications very quickly. However this module is less useful now because Inductive Automation added new functionality which allows users to base their components off of templates. Changing a template changes all components based on that template. Ignition is in continual development with new features and improvements.

Getting Help

There's lots of ways to get help and learn about Ignition. Here's resources I used when implementing Ignition applications:

  • Inductive Automation Forum - When I needed help with something or I didn't know how to do something the Inductive Automation forum was often the first place I went. I would search the forum to see if my question had been asked before or if the answer already exists. I found a lot of useful information this way. The forum is a great place to ask questions. The technical support staff, other users, and the developers of Ignition answer a lot of questions here. This is also a good place to discuss various things about Igniton and share ideas, make feature requests, etc.
  • Inductive Automation phone technical support - For questions and help, particularly when I wanted to know or solve something very quickly.
  • The Ignition User Manual. This is obviously useful for learning about Ignition. I used Appendix C. Scripting Functions as a reference for scripting in Ignition. It shows how to use the built-in Ignition functions in Jython.
  • Google - For when I had a Java or Jython/Python related issue.
  • Java documentation - For when I wanted to understand Java objects or classes in my Jython scripting or Ignition module programming.

The Inductive Automation website includes lots of resources for learning about Ignition. Videos, quick start guides, manuals, articles, example projects, free tools, etc.

Conclusion

I have only given brief descriptions of some of the architecture and functionality in Ignition. It has much more than what I have written about in this post. I didn't write about transaction groups, which is a system for logging data to databases, I didn't write about SQLTags which is a great abstraction for handling data in Ignition. I didn't write about the Jython code editing support, I didn't write about how to develop mobile applications in Ignition and I didn't write about many other things that exist in Ignition modules.

If you want to learn more about Ignition then I recommend reading Steve Hechtman's blog and checking out the Inductive Automation website. You can also download and install the free trial of Ignition.

To get a free live Ignition demo contact Vivian Mudge: vmudge@inductiveautomation.com

I moved back to Sacramento and got a Web development job at Minnick Web Services in 2007. This was really my first full time programming job.

I had been programming a lot for fun in the last few years. It is awesome to suddenly get a full time job to do something you liked doing.

I remember one day in my first week: there was a bug in one of our applications. PHP would error out when downloading some information. I was assigned to fix the problem.

I studied the PHP error and the source code for the application. It seemed to be a memory error. I Googled the error. It seemed like PHP was running out of memory. So I checked the amount of memory PHP was being allocated. It had plenty of memory. I even put some statements in the PHP code to print out how much memory was available so I could be sure PHP had enough memory, and it did. But when running the PHP script to download the information I would still get a PHP memory error. This was a tough problem.

I tried testing the PHP script in a lot of ways to give me clues to what the problem was or where the problem was. Nothing I tried worked until I tried a technique of killing the script in the middle of the source code. If the PHP error was displayed then I would kill the application between the beginning of the script and where I last killed the script. Or if it didn't display the PHP error then I would kill the script between where I last killed the script and the end of the script. Then I would repeat this process. The idea is to find exactly where in the source code the PHP script would error out. I spent a lot of time doing this to find out where exactly it would error out. Interestingly enough I found that the script would error out at different places in the source code, but every time it would error out around a certain function call in the code. So I investigated this function. The function did not come from any user code that was included into the application. The function was not a built-in function from PHP. I found that the function was from a third-party library that was included from the PHP.ini file (which is why I never saw it included into the source code). I found the source code for the function and looked at it. Ah ha! Here it was. For some stupid reason there was a statement in this function that changed the available memory for PHP during runtime to 16MB, causing the PHP script to run out of memory. I just commented out one line of code and the bug was completely fixed.

I spent most of the day trying to fix that bug and I finally did. At the end of the day my boss came to me and asked me what I did for the day. Panic! The only thing I had gotten done in the last 6 hours was comment out one line of source code. I told him that I fixed the PHP memory error bug in the application I was working on.

My boss was surprised. He said, "Really? You fixed it already?". Yes, I did. Much to my surprise he was really happy. He told me that that bug had been in the system for 6 months and nobody could fix it. Nobody had told me about this. I was relieved and became happy too. Certainly programmer productivity is not tied to the number of source code lines written or changed. I would say programmer productivity is how much value you are giving to your employer or customer.

I worked a lot on www.gamasutra.com and related websites adding new features and improving things.

One interesting thing I programmed was the comments. Gamasutra used to not have comments.

I was also asked to show how many comments each news story had on the homepage of www.gamasutra.com and the news index page. Before www.gamasutra.com was redesigned the homepage and the news index page were statically generated. Regenerating the homepage and the news index page each time a user wrote a comment was not an option. So if a webpage is static and you can't regenerate it, how do we update the number of comments for each news story? Implement the feature in Javascript! So I did. I wrote javascript for the static pages which would update the display of the number of comments on each news story. Implementation was fast, it worked well, and it was a slick solution. Here's an image of a bit of the news index page in February 2009 with the number of comments displayed for each news story:

Then there was the Gamasutra.com redesign. That was a big project that I was the lead programmer on. Much of Gamasutra was reprogrammed. I programmed the homepage, news section, features section, job board, contractors section, the AJAX login, the comments, and search. I did the technical part of launching the redesign.

Doing the redesign inspired writing a webpage caching system similar to the caching system in the Smarty templating system. I put the source for the website caching system on Github. It is called SimpleCache.

I did various programming for various gamasutra.com related websites. I was one of the original developers of myGDC, the social website for game developers. I also worked on the online scheduling tool used by the Game Developers Conference, where I used the Javascript framework MooTools a lot.

After Work

Off work hours this was a time of a lot of self learning, exploration of programming technologies and experimentation. 2007 was quite a year for me in self study and side projects. I pretty much exploded with programming technology.

I built a web-based graphics editor. Users could rotate, resize, crop and upload images to be worked on. Here's a blog post about it. I used PHP, Javascript (including AJAX), CSS, XHTML, Imagemagick, and bash scripting to make it.

In June 2007 I bought books on Common Lisp and read them and wrote Common Lisp code. By September I was writing Haskell programs. Here's a blog post about becoming a haskeller.

I went to the very first Bay Area Functional Programmers meeting in San Francisco. I was really excited about Haskell. Here's my blog post about that.

I had been an avid fan of using delicious.com to bookmark web pages around the web for a few years. I had bookmarked probably about 2,000 or more web pages. I would link to my delicious account but one day delicious deleted it.

I liked browsing the web and bookmarking web pages so much that I wrote my own bookmarking web application called newsconomy.com. I built it in 2007 and I still use it for all my bookmarking. I built it for myself but I also built it for others. It has an interesting twist. Instead of just bookmarking web pages, you can trade bookmarks with other people. This is done by "buying" other's bookmarks with an imaginary money called lambda. The idea is to make bookmarking and sharing bookmarks more fun.

I think I made newsconomy too hard for people to understand quickly or I didn't explain it well because nobody else besides myself uses it continually. I also did very little to promote it. I love my bookmarking web application and have 1,857 bookmarks so far. Here's a blog post explaining in detail how newsconomy works. I built it using PHP, MySQL, CSS, HTML.

I became really interested in Javascript after finding and watching Douglas Crockford's videos on Javascript. Also, after reading Crockford's The Little JavaScripter, I bought and started reading The Little Schemer. I love stuff like this.

In 2008 I studied C again. I became interested in the implementation of hash tables in C. I wrote my own implementation of a hash table and did some testing of capabilities of different kinds of hash functions. Here's my blog post about this.

I studied operating systems. For a long time I have desired to work on making a new operating system. Here's blog post I wrote about writing a boot loader in machine code: A Boot Loader in Machine Code.

I have also been interested in the design and implementation of programming languages. This lead me to parsing techniques. Here's an attempt I made on making a Recursive Decent Arithmetic Parser.

In 2009 I went to the first Bay Area Haskell Meeting. I also got a new job working for Calmetrics, the sister company of Inductive Automation.

In my next post I talk about Inductive Automation's Ignition and my work with it.

From 2004 to 2006 I had a job finding, researching, writing/editing technology news stories about state and local governments for www.govtech.com. I would post at least 3 news stories a day. We also made up HTML newsletters of our stories every week and sent them to our readers. We were having some difficulties with our newsletter maker and since I had been learning PHP, MySQL, HTML, Javascript and CSS I thought I would make some utilities to make our lives easier and faster.

It wasn't my job to do any programming so I did it off work hours at first. I wrote a screen scrapper in PHP that pulled our new news off our website and formatted them into an HTML email newsletter. I started using it and I showed it to my boss who liked it and started using it too. This was the start of my professional programming career.

We had another problem. Everyday we spent hours looking through many websites looking for technology news. Same thing every day. This was the same time I had a growing interest in Web programming. In my spare time I was often reading something about programming. I came across an article called Why Python? by Eric Raymond. I loved the article and it got me really interested in Python. So I bought a book on Python and learned it like crazy.

Now back to my problem at work. So much time looking through the same websites over and over. I decided to write a Web crawler to find the news for us. I decided to write the Web crawler in Python! So I started working on this. I made the Web crawler/spider search the most common webpages and websites that we would look through. At first it would search about 25 websites, then 50 then maybe 75 or 100. It would search for technology related content. I showed it to my boss and he liked it. He said that I could spend part of my working hours developing this and I did. I kept tweaking and tweaking and developing it, making it better and better. We called this piece of software the Govbot.

The Govbot found news through a set of regular expressions that I wrote that would analyse the content: basically a complicated keyword search process. But different websites were built differently and I found that while the Govbot would do well searching some websites it would do poorly searching others. So I built a plugin system into the Govbot that made it easy to add code for searching specific websites. That way I could easily customize the Govbot for websites that we wanted to search but that were difficult because of the way they were built. The Govbot showed two categories of results: just new stuff it found and content that was likely to be news of interest to us. My boss continued to use the Govbot for 3 1/2 years after I left my job there.

During my time at govtech I installed and customized a WordPress blog for myself and another blog using Serendipity. I also created a directory of government blogs and a public web-based RSS reader for those blogs.

In 2006 I got a job as a graphics designer in Los Angeles. I mostly designed posters, post cards and emails. I also helped layout a magazine. I used InDesign and Photoshop. While I had this job I found that I missed programming. I found that I would actually prefer using a graphics library to program graphics directly rather than using an authoring tool like Photoshop or InDesign. But I ended up doing some programming anyway. I wrote a plugin in Javascript for InDesign. It would take a list of addresses and a design of a post card and automatically create the post card for each address. This made it much faster to print post cards for each address. I also created a database of contacts using Gadfly, a SQL database system written in Python.

I found out that I am really a programmer, not a graphics designer. But of course I will use GIMP or Inkscape from time to time when I need graphics.

Did I tell you that at this point I had done most of my programming on Linux and that I love Linux? It's true.

In my next post I talk about my first full time programming job.

This is an example of how someone learns Web programming on their own.

My first exposure to programming was in high school. I learned to program using QBasic. How fun was that. My programs ranged from DOS command programs to a spaceship shooter game. For the first time I could see the power I could have over a computer.

After high school I took a C programming course at American River College. I didn't have the prerequisites to be in that course but the instructor let me take the course anyway. I promised myself that I would do well in the course and I did. I remember raising my hand and asking the instructor why we should learn C programming. What benefit would we get from learning it? The professor was actually a bit befuddled by the question and just said we were learning C because it was everywhere. I still didn't know why we were learning C. I later realized that C is indeed everywhere, underlying so much software that we use everyday and influencing so many programming languages that came after. C is like a standard for understanding a certain level of programming.

I guess I was pretty practical. Seems like a lot of students were taking the class because it was another class on a list that they needed to take in order to get their degree. I wanted to learn something that I could use to conquer a piece of the world.

I'm glad I took that C programming class because I learned a lot about programming and about computers. I learned about how computer memory works, pointers, data structures, search and sort and other algorithms. I learned how to write better programs and structure my code. Pretty useful stuff to serve as a beginning foundation for programming.

I got a job doing data entry (got to start somewhere) and awhile later at the same company I got a job researching and editing news for the website www.govtech.com. About the same time I read the book Ender's Game which inspired an idea to start a website called democranet.com (democracy + internet) where people would submit political articles and vote on them to influence popular opinion. I was really excited about doing this but I didn't have anyone to build a website for me and I didn't have any money and I didn't know anything about building websites. So I decided to learn and do it myself.

I asked a friend of mine who did some web development what I should learn to make a website. He told me I should buy books about HTML, CSS, Javascript and PHP. So in 2004 that's what I did. I bought the books and read them and practiced what was in them. I remember following along a PHP book to program a website. It was fun and I learned a lot. Along with PHP I also learned how to use MySQL. I think it wasn't very hard for me to learn Web programming because I already knew how to program.

I actually programmed a lot of functionality of democranet.com, including user registration, user profiles, user-submitted articles, sorting articles, pagination, rating articles, article categories and login. I found a version of www.democranet.com on the WayBackMachine, so you can see and try out this functionality for yourself if you want.

I could program functionality but at the time I didn't know much about making a website look good. Here's a picture:

Later on I made a better design for the website but I never implemented it into the website. What happened was while I was building the website I started to read more political articles and news and I started liking politics less and less, and I found that I was enjoying Web programming and liking it more and more. So I pivoted. I decided to leave democranet.com behind and instead do Web programming.

In late 2004 I came across Damien Katz's mesmerizing blog post Formula Engine Rewrite which hooked me as a reader of his blog, which was a cool thing because I got to read his blog as he first began to work on CouchDB.

In my next post I'll talk about how my professional programming career began.

As a side project I just released a simple free job board here: blueparen.com/jobs.

I wrote it in Clojure and MongoDB. The source code is here: https://github.com/mudgen/jobboard

Since I last posted on my blog I got married, had a baby girl, and quit my job.

I've started a programming consulting business and I am working on a kickstarter project to interview 50 creators of programming technologies.

Here's a picture of my four month old baby girl:

I like a language to allow me to say just what I want to say and lisp in general is geared toward that. The capability of macros is exciting and lends itself towards this as well. I like the core data structures in Clojure, the list, vector, set, and map. I think that a lot of problems are easily solved with just those data structures and Clojure makes it so easy to work with them. There is a power and beauty and capability in a functional style of programming, of using and composing higher order functions. Clojure makes this seem natural and its default immutability with its performance guarantees makes me feel comfortable. I also like Clojure's succinctness and I like closures.
scala, erlang,ruby,clojure,java,python,asp.net Job Trends graph