[MLton] Trip Report -- Commercial Users of Functional Programming

[Stephen Weeks]

On Saturday, September 18, I attended an all-day workshop on
Commercial Users of Functional Programming
(http://www.galois.com/cufp/) in Snowbird, Utah.  There were about 25
attendees and 9 short talks.  All of the speakers save one work full
time in industry and have seen significant use of some functional
language.  There were 2 talks on using Haskell, 3 on OCaml, and 4 on
Scheme.

This was an excellent workshop and I hope it continues.

----------------------------------------------------------------------
Benefits of FP
----------------------------------------------------------------------

All the speakers agreed that their use of a functional language had
given them a large advantage over use of a more traditional language
(Java, C++), and some felt their task would have been almost
impossible otherwise.  Here is an unordered list of the kinds of
things people liked.

* REPL
* type systems 
* rapid prototyping
* ability to code complex algorithms
* using continuations for backtracking
* Scheme macros 
* easy to create a domain-specific language (DSLs)
* good memory management
* performance
* portability

As with most of the audience, I was familiar with these benefits, and
the speakers didn't spend a lot of time on them.  A couple of benefits
that I hadn't heard of before were mentioned by Andy Moran:

* high assurance - a (government) requirement that the code base be
  smaller than a certain size.
* type systems for guaranteeing data separation (the NSA likes this
  kind of stuff) 

----------------------------------------------------------------------
Technical Problems with FP
----------------------------------------------------------------------

The speakers spent a lot of time talking about the kinds of problems
they encountered, both technical and non-technical.  Here's an
unordered list of technical problems.

* difficult FFI
* lack of libraries
* libraries not tuned for performance
* performance
* out of memory errors and memory fragmentation
* lack of platforms
* weak tools (debuggers, profilers)
* lack of formal verification tools
	Here, the programming languages community has shot itself in
	the foot by building better tools for analyzing and verifying
	C/C++/Java than we have for our own languages.
* systems don't work well with Windows
	Bad look and feel, wrong tools, lack of libraries.
* bad interaction with internal build systems and processes
	Byron Cook of Microsoft pointed out that this was a very 
	challenging problem because of so much infrastructure already
	in place
* too much abstraction leading to "encrypted code"
* testing
	Higher-order functions are hard to test.
	Testing infrastructure designed for other languages and
		systems. 
* Haskell issues
	creeping monadery 
		Once you use a monad in one place, it propagates to
		many other places.
	unexplained space behavior 
		A well-known problem with laziness.

----------------------------------------------------------------------
Non-technical Problems with FP
----------------------------------------------------------------------

Most of the speakers found the nontechnical problems to be more severe
than the technical problems.  Here's an unordered list of nontechnical
problems.

* support
	None of these languages have 24-hour support hotlines or much
	available commercial support.  There is no guarantee for fast
	bugfixes.
* sole-source tool
	Most of these tools come from a single supplier, often just
	one or two very important people.  This makes them susceptible
	to the "bus hits Simon (Peyton Jones)" problem that was
	mentioned several times :-).
* lack of long-term viability
* competition with tools developed at company
	(mentioned by Byron Cook at Microsoft)
* lack of product roadmap
* finding and hiring people
	Students don't learn the right things in school.
	People don't see FP as a career path, and don't want to take
		the risk of using it for their job.
* testing
	FP people don't want to test.
	Testers don't want to learn FP.
* convincing managers
* legal issues
	convincing lawyers
	worries about open source (patent issues a'la SCO)
	licensing worries (some companies won't touch the GPL)

There was some agreement that some of the supposed support benefits of
commercial software is illusory, and that open-source addresses it,
but that it still effective in convincing management to use commercial
software.
	
----------------------------------------------------------------------
Some Solutions
----------------------------------------------------------------------

Not too much was offered in the way of solutions.  But here's a few
things that people mentioned to help ease the adoption of FP.

* Grow programmers internally.

* Hire smart people.
	Don't worry about hiring FP experience.  Smart people will
	pick up FP fine.

* Be conservative and successful at the start -- always deliver.
	Management will respond positively if you deliver.  But if you
	fail, even once, they will use that as an argument against FP
	forever.

* hire compiler hackers to provide internal support and a fallback
	plan if the sole-source provider disappears.  A couple of
	companies take this approach.
		Abstrax hired Oscar Waddell (Scheme compiler hacker).
		Galois hired several Haskell compiler people.

* Sell to hardwire designers, who already live in the Unix world
	and understand the value of good tools.

* Extend a standard language with FP features.
	
* When providing languages/interfaces to users, provide multiple
  languages, because different people think differently.

* Form a consortium to finance tools and support.  There was some
  interest in a Haskell Consortium.  Andy Moran also mentioned a
  few problems with the OCaml Consortium:
	1. requires members to attend lots of meetings.
	2. must agree to further the ends of OCaml.
	3. As of 2002, it had 4 members.

----------------------------------------------------------------------
Talk Capsules
----------------------------------------------------------------------

Here is a list of the speakers, where they work, a rough title (or
topic) of their talk, and a brief summary of the talk.  There were no
proceedings, so I may have misspelled some of the names or companies
below.

Andy Moran		
Galois Connections
Using Haskell in an Industrial Context
	Galois has been around since late '99 and focuses on "High
	Assurance Engineering", which does not mean theorem proving,
	but rather giving good evidence that a product does what you
	say it will.

Mieszko Lis		
Bluespec
Using Haskell for a hardware description language
	Bluespec has been around for about 1 year, and has about 20
	people, of which 10 are engineers.  They have about 61K lines
	of Haskell and 109K lines of C (most of the C is a BDD library)

Byron Cook
Microsoft
Using OCaml as part of the Microsoft Kernel Team
	Talked about the Static Driver Verifier,
	http://www.microsoft.com/whdc/devtools/tools/SDV.mspx,
	a tool to prove safety properties of device driver plugins for
	the Windows Kernel.  
	Mentioned that Don Syme is now working full-time on F#, which
	addresses many of the weaknesses of OCaml (lack of C#
	integration, no visual studio integration, licensing).

David Fox
Linspire
Using OCaml at Linspire
	Linspire (formerly Lindows) is a Linux distribution based on
	Debian and targeted at those who don't know or care what OS
	they run.  Linspire has about 60 people, of which 15 are
	engineers and 5 do FP stuff.  They've had a functional
	programming culture from the beginning.  They have some tools,
	including a hardware detection system, written in OCaml.

Nikolaj Bjorner		
Microsoft
A File Replication Protocol implemented with OCaml
	The title says it all.

Kent Dybvig		
Cadence Research and Indiana University
20 years of Chez Scheme
	Talked about Chez Scheme, which had its first release in 1984.
	Chez is closed source. Cadence's business model is to sell
	licenses to big corporations and work under contract to
	provide functionality.  He's considered going open source, but
	has chosen not to because the legal worries will scare away
	users and because sales help to support development.
	
	He talked about Chez's priorities, which are reliability and
	performance, and also "performance continuity", meaning that
	small source code changes shouldn't cause big performance
	changes. 

John Lalonde		
Abstrax
Commercial Grade Scheme
	Used Chez Scheme to build DSLs and combinatorial optimizers for
	build-to-order manufacturing.

Mike Ashley
Beckman Coulter
Using Scheme in End-User Software
	Used Chez Scheme to build a controller for robotic systems
	that run experiments for drug discovery.  They have to present 
	an interface that chemists can use to describe large numbers
	of experiments and provide sensible feedback.

Jonathan Soebel		
SAS Institute
Developing Web Applications with Scheme
	Built a system for producing Java code to let Java programmers
	interface with SQL in a safe way.