Using Stanford NLP to run a sentiment analysis with F#

Stanford NLP is a great tool for text analysis and Sergey Tihon did a great job demonstrating how it can be called from .NET code with C# and F#.

Purpose of this post is to show how StanfordNLP sentiment analysis can be called from F# application. Code used in this example provides sentiment value - from very negative to very positive - for all sentences of the specified text.

Prerequisites:

    -Nuget Stanford.NLP.CoreNLP package needs to be installed (this code works with 3.4.0.0)  
    -Java binaries should be downloaded from http://nlp.stanford.edu/software/stanford-corenlp-full-2014-06-16.zip and unzipped. After that you need to extract content of stanford-corenlp-3.4-
models.jar(it is part of the zip file)to some directory.  

Source code:

    F# code is elegant as usual :)

	open System
	open System.IO
	open edu.stanford.nlp.ling
	open edu.stanford.nlp.neural.rnn
	open edu.stanford.nlp.sentiment
	open edu.stanford.nlp.trees
	open edu.stanford.nlp.util
	open java.util
	open edu.stanford.nlp.pipeline
	let classForType<'t> =
	java.lang.Class.op_Implicit typeof<'t>
	type SentimentPrediction =
	| VeryNegative
	| Negative
	| Neutral
	| Positive
	| VeryPositive
	let classToSentiment = function
	| 0 -> VeryNegative
	| 1 -> Negative
	| 2 -> Neutral
	| 3 -> Positive
	| 4 -> VeryPositive
	| _ -> failwith "unknown class"
	let makeSentimentAnalyzer modelsDir =
	let props = Properties()
	props.setProperty("annotators", "tokenize, ssplit, pos, parse, sentiment") |> ignore
	let currDir = Environment.CurrentDirectory
	Directory.SetCurrentDirectory modelsDir
	let pipeline = StanfordCoreNLP(props)
	Directory.SetCurrentDirectory currDir
	fun text ->
	(pipeline.``process`` text).get classForType<CoreAnnotations.SentencesAnnotation> :?> ArrayList
	|> Seq.cast<CoreMap>
	|> Seq.map(fun cm -> cm.get classForType<SentimentCoreAnnotations.AnnotatedTree>)
	|> Seq.cast<Tree>
	|> Seq.map (RNNCoreAnnotations.getPredictedClass >> classToSentiment)
	|> Seq.toList

view raw sentiment1.fs hosted with ❤ by GitHub

    To call this method you can use following code, where models location should be set to modelsDir variable:

	[<EntryPoint>]
	let main argv =
	let text = "awesome great this text is so exciting! this is disgusting sentence number two.";
	let modelsDir = @"C:\tmp\stanford-corenlp-full-2014-06-16\models";
	let analyzer = makeSentimentAnalyzer modelsDir
	printfn "%A" (analyzer text)
	0 // return an integer exit code

view raw sentiment2.fs hosted with ❤ by GitHub

    Enjoy!

F# - getting function name

Getting F# function name and method info object is tricky but possible:

	let getFunctionName f =
	let type' = f.GetType()
	let method' = type'.GetMethods() |> Array.find (fun m -> m.Name="Invoke")
	let il = method'.GetMethodBody().GetILAsByteArray()
	let methodCodes = [byte OpCodes.Call.Value;byte OpCodes.Callvirt.Value]
	let position = il |> Array.findIndex(fun x -> methodCodes |> List.exists ((=)x))
	let metadataToken = BitConverter.ToInt32(il, position+1)
	let actualMethod = type'.Module.ResolveMethod metadataToken
	sprintf "%s.%s" actualMethod.DeclaringType.FullName actualMethod.Name

view raw gettingFunctionName.fs hosted with ❤ by GitHub

Unfortunately, this code only works when F# compiler does not inline function body into calling method.