My wife recently decided to take on the task of ordering family photo albums. This primarily involved wading through the deluge of digital photos that have been piling up over the last three years, selecting the photos that were worthy to be placed in the album, uploading those photos to a certain web site, organizing the photos in the albums, and finally making the purchase. The particular imaging software, that was used to import/organize/modify the photos, stores the images in a directory structure that looks something like this ..\<Main Directory>\<Year>\<Month>\<Day>. While this would not normally be an issue, it made the task of uploading the images to the photo album creation site quite arduous.
This of course is a simple problem to solve in most programming languages. The following examples show how it could be solved with F# both synchronously and asynchronously. While this is definitely not production ready, it provides a few samples of how asynchronous workflows can be used in the wild.
The Synchronous Way:
(Note: A sleep statement has been added to emphasize the speed difference between the synchronous/asynchronous approaches.)
open System.IO let destinationDirectory = @"C:\temp\picDest\" let sourceDirectoryRoot = @"C:\temp\pic" let searchPattern = @"*.jpg"; let getFileName sourceFile = FileInfo(sourceFile).Name let getSourceImages = Directory.GetFiles(sourceDirectoryRoot, searchPattern, SearchOption.AllDirectories) let getDestinationFileName sourceFile destinationDirectory = destinationDirectory + getFileName sourceFile let copyImage sourceFile destinationDirectory = File.Copy(sourceFile, getDestinationFileName sourceFile destinationDirectory, true) |> ignore do printfn "Starting the image consolidation process with base directory: %s" sourceDirectoryRoot for image in getSourceImages do System.Threading.Thread.Sleep(5000) do printfn "[.NET Thread %d] %s" System.Threading.Thread.CurrentThread.ManagedThreadId image copyImage image destinationDirectory do printfn "The images have been consolidated into directory %s." destinationDirectory do printfn "Press [Enter] close this command prompt." System.Console.ReadKey() |> ignoreThe Asynchronous Way:
(Note: A sleep statement has been added to emphasize the speed difference between the synchronous/asynchronous approaches.)
open System open System.IO type Directory with static member AsyncGetFiles(path:string, searchPattern:string, searchOption:SearchOption) = let fn = new Func <string * string * SearchOption, string[]>(Directory.GetFiles) Async.BuildPrimitive((path, searchPattern, searchOption), fn.BeginInvoke, fn.EndInvoke) type File with static member AsyncCopy(sourceFile:string, destinationFile:string, overwrite:bool) = let fn = new Func<string * string * bool, unit>(File.Copy) Async.BuildPrimitive((sourceFile, destinationFile, overwrite), fn.BeginInvoke, fn.EndInvoke) let destinationDirectory = @"C:\temp\picDest\" let sourceDirectoryRoot = @"C:\temp\pic" let searchPattern = @"*.jpg"; let getFileName sourceFile = FileInfo(sourceFile).Name let getSourceImages imageDirectory searchPattern searchOption = async { return! Directory.AsyncGetFiles(imageDirectory, searchPattern, searchOption) } let getDestinationFileName sourceFile destinationDirectory = destinationDirectory + getFileName sourceFile let copyImage sourceFile destinationDirectory overwrite = async { System.Threading.Thread.Sleep(5000) do printfn "[.NET Thread %d] %s" Threading.Thread.CurrentThread.ManagedThreadId sourceFile return! File.AsyncCopy(sourceFile, getDestinationFileName sourceFile destinationDirectory, overwrite) } do printfn "Starting the image consolidation process with base directory: %s" sourceDirectoryRoot let sourceImages = getSourceImages sourceDirectoryRoot searchPattern SearchOption.AllDirectories |> Async.RunSynchronously for image in sourceImages do copyImage image destinationDirectory true |> Async.Start do printfn "The images will be consolidated into the following directory: %s" destinationDirectory System.Console.ReadKey() |> ignore
A few questions:
ReplyDelete- What's the advantage of having getSourceFiles be async?
- What is the exact interaction between Async.Start and the threadpool -- I assume Async.Start will block if there are no threadpool threads available? Even so, this is probably too many threads, no?
You mention hilighting the speed differrences between the approaches, but make no mention of them - what were your findings? (Without the sleep() - that's cheating :))
I'd expect the async copy to not be much, or any faster, as the copy operation is I/O bound, and a busy i/o channel and added disk head movement could actually slow things down.
thanks
New here,
ReplyDeleteI'm just browsing sites for the children of Haiti.
I'm doing my part for a non-profit organization that gives time to
creating an oppurunity for the kids in haiti. If anybody wants to give money then this is the place:
[url=http://universallearningcentre.org]Donate to Haiti[/url] or Help Haiti
They provide children in Haiti a positive learning environment.
Please check it out, they're real.
Please give
Genial post and this enter helped me alot in my college assignement. Thanks you for your information.
ReplyDelete[COLOR="Red"][B]Click on the pictures to view in full size[/B][/COLOR]
ReplyDelete[URL=http://www.wallpaperhungama.in/details.php?image_id=12779][IMG]http://www.wallpaperhungama.in/data/media/32/Asin-80.jpg[/IMG][/URL]
[URL=http://www.wallpaperhungama.in/details.php?image_id=7237][IMG]http://www.wallpaperhungama.in/data/media/32/Asin-71.jpg[/IMG][/URL]
[url=http://www.wallpaperhungama.in/cat-Asin-32.htm][b]Asin Hot Wallpapers[/b][/url]
Photo gallery at t WallpaperHungama.in is dedicated to Asin Pictures. Click on the thumbnails into enlarged Asin pictures, intimate photographs and exclusive photos. Also after to other Pictures Gallery for Turbulent quality and High Acutance twin scans, talking picture captures, movie promos, wallpapers, hollywood & bollywood pictures, photos of actresses and celebrities
Hello,
ReplyDeleteI have developed a new clean web 2.0 wordpress theme.
Has 2 colours silver and blue, has custom header(colour or image).
I am curently working on it, so if you have suggestions let me know.
You can view live demo and download from here www.getbelle.com
If you found bug reports or you have suggestions pm me.
Wish you a happing using.
many thanks to [url=http://www.usainstantpayday.com/]USAInstantPayDay.com[/url] for paying the hosting and developement of the theme
Fumboppophy
Hello friend amazing and very interesting blog about F# - Asynchronous Directory.GetFiles and File.Copy
ReplyDeleteHello I want to congratulate to them by its site of the Web of the excellent looks like entertained and very good very to me it elaborated.
ReplyDeleteHello I want to congratulate to them by its site of the Web of the excellent looks like entertained and very good very to me it elaborated.
ReplyDelete