hubFS: THE place for F#

Based on the title, you may guess that I work in finance.  I do, though my UG is in Computer Science.  I find that I learn best by implementing things and then experimenting with them, so I wanted to learn about Stats/High Frequency/AI technologies by taking the same approach. 

What I'd like to understand is whether F#/ML is well suited to this sort of analysis.  I don't know ML very well (used it for a course back in 2000) and am wondering if there are benefits to using it for this sort of project.  If there are real benefits, then it makes sense to spend the time to learn it while I code up these systems.

Any thoughts on the matter?  While I'm sure it has been discussed, I am also looking for experiences with respect to performance.

Great, there goes the neighborhood.  Who let the finance guys in?  Seriously, I've been known to do a few things in quant finance, so I guess I can give an opinion.

Yes, F#/ML is well suited to all three disciplines that you mention.

• Statistics - there aren't a lot of libraries out to do this type of work, but implementation is cheap for many of the basics.  There are also a few good libraries (dnAnalytics, QuantLIB, etc.) that have great code bases (not in F#) that can either be used directly (interop with F#), or as a model for building your own.  Obviously those libraries are also focused on aspects in Quant Finance and not just the statistics aspect.  I still use 'R' for some of my statistics works, but I'm trying to get as much built in F# as necessary to cover that functionality.
• High Frequency - Give me your view on "high-frequency".  Are you talking real time tick data off of standard finance feeds?  I keep about 2TB of daily data from the markets and find F# as one of the best manipulation tools.  I think that SQL Server Service Broker and/or WF present useful models of infrastructure although neither is a feature rich as BizTalk.  F# plays well with all of these and adds suitable advantage.
• AI - I currently have models within JJB Research to perform Genetic Algorithms, Genetic Programming, Emergent Computation, Swarm Computation and ACO (Ant Colony Optimization) all in F#.  I'm just starting to do work in Bayesian Networks, SVMs and a few other disciplines.  I have not done any significant work in Neural Nets in F# although that would be trivial to implement.  While I can't share those code bases, I can tell you that F# adds value in each of these areas.

I tend not to think of "vs." when doing this type of work.  F#, C#, C++ and others are nothing more than tools.  Used well, each is capable of yielding great results.  For me the key is the interop, i.e. I want an environment where I can do F#, C#, SQL Server, BizTalk, WF, WCF, XML, etc. without regard to costs of interop.  I get that with the .NET runtime.  All of those other tools make things easy to build.  F# is the key to my infrastructure for compute intensive tasks.  In representation, it is less verbose and much easier to represent many of my works, models in particular, when compared with any other language out there, even outside of .NET.

What you should consider is the cost in time to learn.  Be certain that you have enough time/effort to learn these languages to a level of competence that will give you the leverage that you need.  If you already have ML in your background, I would think the cost to learn F# would be cheap.

 

There is not much that I can add over the experiences and opinion expressed by optionsScalper. 

I have been working on AI for the last 10 years and have used and tried a lot of programming languages over this time including C/C++/C#/Java/JavaScript/MATLAB/R/S/S+. I am now using F# most of the time, in particular when it comes to non-linear data analysis (MATLAB is a good vehicle when all you are doing is linear model analysis and the datasets are only moderately sized).

The three major advantages I see for using F# are:

1. Succinctness: For example, I have recently ported a factor graph inference library from C# to F# using language concepts that are only availabel in F# ( for example, object expressions ) and have shortened the code by a factor of 5. 
2. Performance: I do not notice any performance difference between C# and F#. In my job, I often perform a very detailed data analysis of 100's of GB of Xbox Live log data and performance is as good as it can get.
3. .Net Intereprability: The possibly biggest advantage of F# is that it integrates deeply with .Net. The power of F# lies in the fact that you get the beauty of a functional programming language with the whole CLR at your fingertips. In the above mentioned data analysis I work with SQL 2005 servers and the whole task of getting the data from the server takes a staggering 15 lines (I am sure this can be improved and I would like to hear other people's opinion ).

As an example, here is the above mentioned SQL reading code

open System
open System.Data.SqlClient

///Creates a SQL connection for a database.
let createSqlConnection database =
 let sqlConnection = new SqlConnection ("Initial Catalog = " ^ database ^ "; Data Source = testserver; Integrated Security = SSPI") in
 sqlConnection.Open();
 sqlConnection

///Creates a SQL data reader iterator.
let sqlIterator statement connection =
 let command = new SqlCommand (statement, connection) in
 command.CommandTimeout <- 0;
 command.ExecuteReader()
 |> IEnumerable.unfold (fun (reader:SqlDataReader)  -> if (reader.Read()) then Some(reader,reader) else (reader.Close(); None))

 

Hope this helps your decision making,

Ralf Herbrich

Applied Games Group, Microsoft Research Ltd., Cambridge, UK

Thanks for the comments guys.  What's so bad about the "finance guys" anyhow?  I'm barely a finance guy, though I want to learn the stuff very well, hence the idea to write some of my own models. :o)

Regarding High Frequency, I am referring to the analysis of tick data. Frankly I don't even know what to do with it if I had it around right now, but it's definitely something I want to work toward.

Scalper, what kinds of stuff do you do with quant finance?