Showing posts with label Querly. Show all posts
Showing posts with label Querly. Show all posts

Wednesday, May 5, 2010

Integrating F# and Erlang Using OTP.NET

Recently, I ran across a post that talked about a .NET library named OTP.NET.  OTP.NET is a port of a Java library named JInterface that allows a .NET application to act as an Erlang node.  OTP.NET hasn't been updated in a while and the API is a bit clunky, but the provided functionality is very cool.  Note: A forked version of OTP.NET that has been migrated to VS2010 can be found at http://github.com/dmohl/OTP.NET-2010.

I decided to give OTP.NET a test drive by adding a simple command-line interface to an Erlang project that I have been playing with called Querly.

Querly is a simple query engine that allows T-SQL style queries against a CouchDB database.  You can read more about Querly in the following posts:

- Querly - A Query Engine for CouchDB
- Setting Up Querly
- Getting Started with Querly: A Simple CouchDB Query Engine

Getting Setup


To make this work, we first need to do the following:

1. Setup Querly by following the instructions provided in the blog post entitled Setting Up Querly.
2. If you want to load up a database for testing, follow the instructions provided in the blog post entitled Getting Started with Querly: A Simple CouchDB Query Engine.
3. Make sure that RabbitMQ and CouchDB are running.
4. Launch Querly by navigating to the Querly directory and running a command such as:
"C:\Program Files\erl5.7.5\bin\werl.exe" -sname querly -setcookie supersecretcookie -pa ./ebin -pa ./src
5. Finally, in the Erlang interactive shell, run the following command:
querly:start().

If everything worked as expected, you should see something like this:

Building the F# Application

Now that we have the Erlang application up and running, it's time to put together the F# code that will allow us to join the Erlang node cluster and interact with Querly.  Note: You'll need to replace the value assigned to peerNode with the name of your querly node. The full solution can be found at http://github.com/dmohl/Ferly.
module Ferly

open System
open Otp

Console.Write ":>"  
let mutable input = Console.ReadLine()  

let erlangCookie = "supersecretcookie"
let peerNode = "querly@dmohl-PC" // Replace with querly node name

let connection =
    let cNode = new OtpSelf("querlyclientnode", erlangCookie)
    let sNode = new OtpPeer(peerNode)
    cNode.connect(sNode)

let RemoteCall (sql:string) = 
    let arguments = [|new Erlang.String(sql)|] : Erlang.Object[]
    do connection.sendRPC("querly_client", "sql_query", arguments) 
    connection.receiveRPC().ToString()

while input <> "quit" do 
    Console.Write(RemoteCall input)
    Console.Write "\n\n:>"   
    input <- Console.ReadLine()


We should now be able to run the F# application and write queries against a couch database.  An example of the running application after a few processed queries is shown below:


Wrapping Up


We have only touched the surface of the functionality provided by OTP.NET.  Hopefully this post has shown how OTP.NET can be used to quickly integrate F# with Erlang.  Unfortunately, there is not a lot of documentation out there for OTP.NET, so if you want to learn more, your best bet is to pull down the code and start thumbing through it.  Happy polyglot programming!


Sunday, May 2, 2010

Getting Started with Querly: A Simple CouchDB Query Engine

About a month ago, I introduced a simple project that allows limited T-SQL style queries against a CouchDB database.  The codename for that project is Querly.  Today, I'm going to talk a little about what you need to get started with Querly.

The other posts can be found in the following locations:

- Querly - A Query Engine for CouchDB  
- Setting Up Querly

Loading Up A Test Database

To get started with Querly, make sure that you have followed the instructions outlined in Setting Up Querly.  If all the tests run, you are ready to get started with Querly.  Press [Enter] to get back to the Erlang prompt.  In order to load up a database with several documents that we can play with, type  querly_load_tests:load_10000_people(). and press [Enter] (Note: This will take a little while to finish).

Writing Our First Query

Once the "person" database is loaded, we can start Querly and begin writing simple queries.

To start querly, type querly:start().

Here are a few samples of queries that can now be run (Note: The first query may take a few seconds as the documents are being loaded into an ETS table):

querly_client:sql_query("select * from person where Idno = 1").
querly_client:sql_query("select * from person where FirstName = TestFirst2").
querly_client:sql_query("select * from person where Dob= 08/28/1977 and FirstName = TestFirst3").

Querying Other Databases

In order to write T-SQL style queries against a database, a record must be defined in the src/record_definitions.hrl file.  An example for the person table is as follows:

-record(person, {'Idno', 'FirstName', 'LastName', 'Dob', 'Ssn'}). 
-define(personFields, record_info(fields, person)).

This definition makes it possible to use specific field names in the TSQL style queries.

Limitations

There are a number of known limitations in this simple query engine. 

The most notable limitations are as follows:

- The T-SQL style query functionality is limited to simple "where" clauses.  
- A record must be predefined in order to query a database.

Conclusion

Querly allows simple T-SQL style queries against a CouchDB database.  While Querly has a number of known limitations, it hopefully serves as a simple PoC and/or starting point for a CouchDB query engine.

Wednesday, March 31, 2010

Setting Up Querly

In my last post, I introduced Querly (a simple CouchDB query engine).  In this post, I provide the steps for setting up Querly.

1. Start by installing CouchDB.  A windows installer is available on the couchdb wiki.
2. Next, install Erlang.  I'm using version R13B04.
3. Now, install RabbitMQ, available at http://www.rabbitmq.com/download.html.  Note: Querly has built in support for subscription to a RabbitMQ queue.  This allows updates to be received without having to reload an entire database from CouchDB.  This is not required for utilization of the basic functionality of Querly; however, it is required for auto update support and for all of the tests to run successfully.
4. Launch CouchDB and RabbitMQ.
5. Retrieve Querly from http://github.com/dmohl/Querly.
6. Edit make.bat in the Querly directory and change the Erlang directories appropriately.
7. Launch Querly by executing the batch file in the main directory.  This will run all of the tests.

In a future post, I'll talk about how to get started with Querly.

 

Thursday, March 25, 2010

Querly - A Query Engine for CouchDB

I've been playing with Erlang and CouchDB for a little while now and have been completely amazed by both.  One of the things that would be very helpful for my team to be able to use CouchDB in more production scenarios would be to have enhanced query functionality.  Querly is the first attempt at this enhancement.  Querly is a solution written in Erlang that allows ETS and/or limited SQL style queries against CouchDB.  While there are a number of limitations with this first attempt, it provides a starting point.  The code is available at http://github.com/dmohl/Querly.

I plan to provide additional information including setup instructions, features, limitations, etc. over my next few blog posts.