Thursday, April 1, 2010

.NET Remoting in F#: A Start to a Distributed Hash Table (DHT)

Over the last few weeks I've been talking with a few guys on my team (@a_robson@therealhoff@elijahmanor@ifandelse) about Distributed Hash Tables (DHT).  A DHT seems like a fun thing to build that would be a good fit for F#.  


I decided to look at starting down the path of implementing the Chord protocol (http://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf).  The first problem that needed to be solved was that of communication between the nodes.  The ideal solution seems to be the model provided by Erlang.  Unfortunately, that model doesn't seem to be available in .NET land (there is MPI.NET, but that appears to require Windows HPC Server 2008 or Microsoft Compute Cluster Server 2003 for use across multiple machines).  After some research, I finally decided on using .NET remoting (though I may end up changing this to a sockets based approach in the future).  The POC for this is provided below.  


Seeing it work: 


1. First, run Polyphony.exe from the bin\Server directory. 
2. Next, run Polyphony.exe from the bin\Debug directory.




3. Now, type "put 1 test1" in one of the consoles and "put 2 test2" in the other.



4. Now, type "get 1" in either console.  The app. will first check it's local hash table.  If the value is not found, it moves on to the second node.






The next step is to make this scale and more robust.  Then it's on to some of the specific Chord protocol functionality.  The source is available at http://github.com/dmohl/Polyphony and will be continually evolving.  Let me know if you want to participate in the fun.


Example:

Program.fs
module Polyphony

open System
open System.Configuration

let port = ConfigurationManager.AppSettings.Item("ServerPort")

ChordServer.Initialize(Convert.ToInt32(port)) |> ignore
ChordClient.Initialize() |> ignore

Console.Write "\nEnter Command:"
let mutable input = Console.ReadLine()
 
while input <> "quit"
    do
    if input <> "quit" 
    then
        ChordClient.RunCommand input
        Console.Write "\nEnter Command:" 
        input <- Console.ReadLine()
Chord.fs
module Shared

open System
open System.Collections
open System.Runtime.Remoting

type Chord = class
    val hashTable : Hashtable
    inherit MarshalByRefObject 
    new () = {hashTable = new Hashtable()}
    member x.PutValueByKey key value =
        x.hashTable.Add(key, value)
    member x.GetValueByKey key =
        x.hashTable.Item(key)
end                
ChordServer.fs
module ChordServer

open System
open System.Runtime.Remoting
open System.Runtime.Remoting.Channels
open System.Runtime.Remoting.Channels.Tcp

let Initialize port =
    let tcpChannel = new TcpChannel(port)
    Console.WriteLine("Polyphony Server started on port {0}...", port)    
    ChannelServices.RegisterChannel(tcpChannel, false) |> ignore
    let commonType = typeof<Shared.Chord>
    RemotingConfiguration.RegisterWellKnownServiceType(commonType,
        "chord", WellKnownObjectMode.Singleton) |> ignore
ChordClient.fs
module ChordClient

open System
open System.Configuration
open System.Runtime.Remoting
open System.Runtime.Remoting.Channels
open System.Runtime.Remoting.Channels.Tcp

let serverInformation = ConfigurationManager.AppSettings.Item("ClientServer")
let selfServerInformation = ConfigurationManager.AppSettings.Item("SelfServer")

let Initialize() =
    Console.WriteLine("Polyphony Client started and pointing to server {0}...", serverInformation)    
    let localObject = Activator.GetObject(typeof<Shared.Chord>, selfServerInformation) :?> Shared.Chord 
    localObject.PutValueByKey "test" "testValue"

let RunCommand(input:string) : unit =
    
    let inputArguments = input.Split(' ')
    let result = 
        match inputArguments.[0] with
        | "put" -> 
            let localObject = Activator.GetObject(typeof<Shared.Chord>, selfServerInformation) :?> Shared.Chord     
            localObject.PutValueByKey inputArguments.[1] inputArguments.[2] |> ignore
            sprintf "PUT Key:%A Value:%A" inputArguments.[1] inputArguments.[2] :> obj   
        | "get" -> 
            let rec getValue server =
                let chord = Activator.GetObject(typeof<Shared.Chord>, server) :?> Shared.Chord         
                match chord.GetValueByKey inputArguments.[1] with
                | null -> getValue serverInformation    
                | value -> value
            getValue selfServerInformation
        | _ -> "unknown command" :> obj   
    Console.WriteLine(result) |> ignore
App.config
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="ServerPort" value="9877"/>
    <add key="SelfServer" value="tcp://localhost:9877/chord"/>
    <add key="ClientServer" value="tcp://localhost:9876/chord"/>
  </appSettings>
</configuration>


8 comments:

  1. Nice article... BUT!

    In 2010 - *don't* use .NET Remoting. Please.

    I'd suggest you use WCF with one of the binary bindings (netTcpBinding / netPipeBinding).

    --larsw

    ReplyDelete
  2. :) Point well taken! I will definitely be moving to either WCF or a sockets based approach.

    Thanks for the comment.

    - Dan

    ReplyDelete
  3. By the way, MPI.Net does not require running HPC, though you will need
    "Microsoft HPC Pack 2008 SDK" for the MSMPI implementation and utilities (notably, Microsoft's versions of smpd and mpiexec).
    Once you have that and MPI.Net, you can build and run your distributed apps using mpiexec to specify which nodes run which processes, and smpd running as the process manager demon on each node (which can be 32 or 64 bit OS).
    Of course then, life get interesting ...
    David

    ReplyDelete
  4. I would like to say that you really made my day, it's wonderful when you just look around the web
    and find something like this, reminds me of that ''How to make a dinner for a romantic...'' by Elsa Thomas,
    you're a wonderful writer let me tell you!!! ñ_ñ

    James Maverick (maverickhunterjames@gmail.com)
    3453 Rardin Drive
    San Mateo, CA 94403
    Project Manager
    650-627-8033

    ReplyDelete
  5. By the way, MPI.Net does not require running HPC, though you will need
    "Microsoft HPC Pack 2008 SDK" for the MSMPI implementation and utilities (notably, Microsoft's versions of smpd and mpiexec).
    Once you have that and MPI.Net, you can build and run your distributed apps using mpiexec to specify which nodes run which processes, and smpd running as the process manager demon on each node (which can be 32 or 64 bit OS).
    Of course then, life get interesting ...
    David

    ReplyDelete
  6. :) Point well taken! I will definitely be moving to either WCF or a sockets based approach.

    Thanks for the comment.

    - Dan

    ReplyDelete