ch.rgw.io
Class ExternalSort

java.lang.Object
  ch.rgw.io.ExternalSort

public class ExternalSort
extends java.lang.Object

Goal: offer a generic external-memory sorting program in Java. It must be : - hackable (easy to adapt) - scalable to large files - sensibly efficient. This software is in the public domain. By Daniel Lemire, April 2010 http://www.daniel-lemire.com/

Constructor Summary

ExternalSort()

Method Summary

static void	main(java.lang.String[] args)
static int	mergeSortedFiles(java.util.List<java.io.File> files, java.io.File outputfile, java.util.Comparator<java.lang.String> cmp) This merges a bunch of temporary flat files
static java.io.File	sortAndSave(java.util.List<java.lang.String> tmplist, java.util.Comparator<java.lang.String> cmp)
void	sortFile(java.io.File in, java.io.File out, java.util.Comparator<java.lang.String> cmp)
static java.util.List<java.io.File>	sortInBatch(java.io.File file, java.util.Comparator<java.lang.String> cmp) This will simply load the file by blocks of x rows, then sort them in-memory, and write the result to a bunch of temporary files that have to be merged later.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExternalSort

public ExternalSort()

Method Detail

sortInBatch

public static java.util.List<java.io.File> sortInBatch(java.io.File file,
                                                       java.util.Comparator<java.lang.String> cmp)
                                                throws java.io.IOException

This will simply load the file by blocks of x rows, then sort them in-memory, and write the result to a bunch of temporary files that have to be merged later.

Parameters:

file - some flat file

Returns:

a list of temporary flat files

Throws:

java.io.IOException

sortAndSave

public static java.io.File sortAndSave(java.util.List<java.lang.String> tmplist,
                                       java.util.Comparator<java.lang.String> cmp)
                                throws java.io.IOException

Throws:

java.io.IOException

mergeSortedFiles

public static int mergeSortedFiles(java.util.List<java.io.File> files,
                                   java.io.File outputfile,
                                   java.util.Comparator<java.lang.String> cmp)
                            throws java.io.IOException

This merges a bunch of temporary flat files

Parameters:

files -

output - file

Throws:

java.io.IOException

main

public static void main(java.lang.String[] args)
                 throws java.io.IOException

Throws:

java.io.IOException

sortFile

public void sortFile(java.io.File in,
                     java.io.File out,
                     java.util.Comparator<java.lang.String> cmp)
              throws java.io.IOException

Throws:

java.io.IOException