The Origins of the Graphical User Interface

In this series of articles, I will be going through the history of the Graphical User Interface (GUI). The questions addressed will include:

  • What is a GUI?
  • How old is it?,
  • Who created the first one?
  • Why did they create it?
  • How have they evolved over time?

I will try to cover as much of this as I can.

What is a GUI? A GUI (pronounced gooey) is how you interact with a computer or other electronic device without needing to type in commands using a keyboard. The most common ways of interacting are by way of a stylus, mouse/trackball, touch pad or just your finger. You tap or click on icons that represent functions you wish to perform. When the early computer pioneers created text input via keyboards, the computer become more broadly used. The early adoption was slow and had a niche following. They knew a better way was needed.

How old is it? The idea started longer ago than you might think. It all started with a device called the light pen. The light pen was created around 1955 at the Massachusetts Institute of Technology (MIT), a private land-grant research university in Cambridge, Massachusetts. This was part of a larger project. The U.S. Navy’s Naval Research Lab approached MIT about creating a computer to act as a flight simulator to train bomber crews. After some initial talks they decided to fund the development under the name Project Whirlwind.

The light pen was a light-sensitive wand that was connected to a computer terminal’s cathode-ray tube (CRT) display. The light pen detects changes in brightness of screen pixels when updated by the CRT’s electron beam and sends that timing information to the computer. Since the update of pixels was one at time by the electron beam and at a known refresh rate, the computer terminal could figure out where the light pen was touching the screen. Over time, once it was determined to be reliable, it was more widely used on the Situation Display consoles of the AN/FSQ-7, a large computer system for military airspace surveillance built by IBM (though MIT still aided in the development). After IBM took over development, they made use of this technology on many of their other terminals.

A few things about the terminology and the technology that was used in the 1950’s and 1960’s: These computer systems were multiple cabinets that would fill an entire floor of a building. When they say it was a graphics terminal, by today’s standards it was text . These Graphics Display Units (terminals) were only input and output devices, as they needed to connect to a Central Processing Unit to be of any use. The graphics terminals themselves processed all the input and output for the Central Processing Unit. A graphics terminal consisted of a CRT, light pen and a keyboard. To show an example, here is an IBM 2250 Graphics Display Unit, click here.
The Central Processing Unit is what we would refer to as the computer. It contains the core memory, disk storage, keyboard, control panel and a line printer. You can click here to see an example of a IBM 1131 Central Processing Unit and here for an example of a IBM 1130 Central Processing Unit. Click here to see them connected together. (Images courtesy of Columbia University in NYC, IBM, The Museum of Applied Arts & Sciences and By Martin Skøtt – Flickr: IBM 1130, CC BY-SA 2.0 via Wikimedia)

Countless people at many companies and universities continued further development throughout the 1960’s. The US Government continued their involvement by funding many of these projects. Here are some notable creations:

The Sketchpad was a computer program by Ivan Sutherland in 1963. It is considered to be the ancestor of both Computer Aided Design (CAD) and a major breakthrough in computer graphics in general. It was the first program to use a complete graphical user interface. It ran on the Lincoln TX-2, which was an advanced transistor-based computer built in 1958 by MIT’s Lincoln Laboratory. Users would draw on the screen using a light pen. They could create an initial drawing and then further modify it using geometric constraints. Geometric constraints was a major invention. In short, this allowed for precise drawing as apposed to free hand drawing. It would correct crooked lines to an exact length, and angles could be maintained while the drawing is modified. Below is a 7 minute 16mm film from 1963, it is a demonstration of how Sketchpad works.

The RAND Tablet, developed by the Rand Corporation in 1964, was a input device that used a stylus attached to a 10″ by 10″ (254 mm by 254 mm) printed-circuit screen using capacitive sensors. It looks and functions similar to today’s drawing tablets, like a Wacom tablet. The Rand Tablet had 100 lines per inch (25.4 mm) resolution, capable of digitizing 1 million locations, and used a handwriting recognition program called GRAIL (Graphical Input Language). GRAIL could identify 53 hand drawn numbers, letters, symbols and geometric shapes. It was also able to use gestures to manipulate text and etc on the screen. This product was the inspiration for the later creation of devices like Wacom’s drawing tablet, Apple’s Newton and the Palm Pilot by Palm Inc..

RAND continued to develop computing devices and computer systems. They also partnered with IBM to create the VGS (Videographic System). It was a very interesting computer system, but a little too much to explain here. It combined video content (viewable on modified TV’s) and the RAND Tablet to allow a way of annotating the content. Cartographers found this system an amazing tool to create and annotate maps. I will link some PDF files below.

RAND Videographic System. This is a technical over view of the complete system. Shows all the components that were used.

RAND and the Information Evolution. For information specifically on the Videographic System, start at page 89.

Light pen usage continued into the 1980’s, as early personal computers had the ability to use them. They fell out of use later in the decade, due the fact that you needed to keep your arm extended for long periods of time. A similar technology was used on game consoles, where the light pen was made to look like a gun, such as was used in a popular arcade game called Duck Hunt.

Shown below are two YouTube videos on how the RAND Tablet functioned. The first one is a 2 minute excerpt of a 97 minute presentation done by Alan Kay called “Doing With Images Makes Symbols: Communicating With Computers”. He is one of the many computer scientists that were responsible for the graphical interfaces that we use today, and is best known for is work on object-oriented-programming and windowing graphical user interface design. The second video is the original video Alan was commenting on in his presentation. Alan started it at the 7 minute and 30 second mark. The entire video is 14 minutes long and was produced by the RAND Corporation from the mid 1960’s. It is a very well produced video and the computer screen is very clear and legible. If you don’t want to take the time watching it; I would suggest at least skipping through it, as I think you will be surprised at its functionality.

Alan Kay “Doing With Images Makes Symbols: Communicating With Computers”.

The last two projects of the 1960’s I am going to mention were also revolutionary in how modern graphical user interfaces operate. The following was the work of Douglas Engelbart, he was an engineer and inventor. He joined the Stanford Research Institute (now know as SRI International) in 1957. His interests centered around how people interact with objects they use in day to day activities, with a particular interest in how children do the same. He recruited others in the 1960s to form the Augmentation Research Center at the Stanford Research Institute. Out of the many contributions he and his group made to computing, I will mention three of them that I consider ground breaking.

He created the idea of hypertext. It is the foundation of how point and click actually works. The icon is a link that refers to another object. Professor Sir Tim Berners-Lee, a Computer Scientist at CERN (European Nuclear Research), used this idea to create HTML in 1989, and released it in 1990. To quote him “I just had to take the hypertext idea and connect it to the Transmission Control Protocol and domain name-system ideas and—-ta-DA!—the World Wide Web”.

Continuing on with Douglas, he and his group (he is named on the patent) created the computer mouse. It was given that as a nickname because the cord resembled a mouse tail. They intended to create an official name but the nickname caught on. The first prototype was in 1964 with its public demonstration in 1968. The patent was filed on 1967 and granted in 1970. You might find it interesting that the trackball predated his invention. It was invented by two separate people in separate countries a few years apart. Going by the earliest date first, an English engineer named Ralph Benjamin invented it in 1946 and was granted the patent in 1947, he called it roller ball. A few years later in 1952 a British electrical engineer in Canada named Kenyon Taylor and his team at the Royal Canadian Navy’s Digital Automated Tracking and Resolving project (DATAR) created their version of the trackball.

I saved the best for last. One of the three aforementioned projects Douglas worked on was the Augmentation of Human Intellect project. That project created the oN-Lin System (NLS). Douglas was inspired by some papers written by Vannevar Bush in 1945. He was an early engineer, inventor and science administrator for the US Government. His concept was called Memex. It outlines how our computers work today, really shocking how he described the functionality 76 years ago.

The NLS was software that allowed many functions. To name a few, there was document creation/removal and editing. You could select text and link it to other portions of the document or even another document (a form of hypertext linking). Edit a document with multiple people remotely, which makes it the first groupware application. (Think of it as a Wiki.) This had the ability to move and resize multiple windows. It was also the first time anyone had ever seen a mouse before. The early development was using the CDC 3100 in 1965 and later using the Scientific Data Systems (SDS) 940 in 1968. These were smaller yet more powerful computers compared to those used in the 1950’s. Instead of filling a floor or multiple floors of a building, these computers (fully connected) were about size of a bus. Each unit was about 4 ft wide (1.14m) and 6 ft tall (1.87m). The amount of these connected together determined the computation ability and the amount of memory available, for a lack of a more technical explanation.
Developments of design concepts began and were supported by the US Air Force from 1959 to 1960, and Douglas later published a framework in 1962. In 1968, development moved to the above mentioned SDS 940 using the Berkeley Timesharing System, which was an operating system that allowed multiple people to use it. Non time-sharing operating systems would need to process the request of person before it could process another.

Douglas and his team gave a demonstration at the Association for Computing Machinery / Institute of Electrical and Electronics Engineers (ACM/IEEE)—Computer Society’s Fall Joint Computer Conference in San Francisco, California on December 9th, 1968. This demonstration is referred to as The Mother of All Demos. Douglas’s terminal was connected via a home built 1200 baud modem back to their lab in Menlo Park. Additionally a mouse, keyboard and a 5 key chorded keyset was connected to the terminal. They also had two microwave transmitters, video switcher and cameras on loan from NASA. Which providing video between the conference and their lab in Menlo Park. Output of terminal was projected on a large screen for the audience. After the 90 minute long demonstration the audience gave a standing ovation. I will link below the 90 minute video and a highlights playlist of 10 excerpts plus 2 additional videos from the 90 minute video. It is about 24 minutes if you watch all 12. I found it quite amazing at what this software could do. There is an over the shoulder view of the keyset, mouse and keyboard. The keyset in on the left, it looks like a 5 key mini piano.

The Mother of All Demos
Excerpts of from The Mother of All Demos.

To see the list of videos click once on the = in the top right of the video window, if it does show them click again.

In the next article, we will enter into the 1970’s and the beginning of the GUI as you know it today.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.