Re: [SLUG] Slicing a file

From: Paul M Foster (paulf@quillandmouse.com)
Date: Fri May 26 2006 - 02:17:52 EDT


Eben King wrote:
> On Fri, 26 May 2006, Paul M Foster wrote:
>
>> Here's what I need to do (sometimes): Take a text file and split it
>> into pieces. The split command won't work for this, because the pieces
>> may not be equal. For example, I may want to split an 8000 line file
>> into pieces 2000, 3000 and 3000 lines each. I'm willing to run the
>> same command multiple times with different parameters, but whatever I
>> use needs to be some common command expected to be there on any
>> system. I've looked at split, head and tail. I suspect I can do this
>> with sed or awk, but I'm not very familiar with those two, beyond
>> rudimentary use. Can anyone suggest a solution?
>
> Find out what line is the first that should be in the second file (call
> it L).
>
> head -n L-1 filename > file1
> tail +L filename > file2andLater
>

(Note: I'm not using a second file to tell me where the splits should be.)

A (real) example of the above would be (assuming original file contains
10000 lines, and needs splits of 2000, 3000, 3000 and 2000):

head -n 2000 original > result1
tail -n 8000 original > temp1
head -n 3000 temp1 > result2
tail -n 5000 temp1 > temp2
head -n 3000 temp2 > result3
tail -n 2000 temp2 > result4

The above will work, but leaves temp files potentially all over the
place (temp1 and temp2). If necessary, I can do that. But it would be
better to create the exact "chunks" from a file without any extra files
lying around.

-- 
Paul M. Foster
-----------------------------------------------------------------------
This list is provided as an unmoderated internet service by Networked
Knowledge Systems (NKS).  Views and opinions expressed in messages
posted are those of the author and do not necessarily reflect the
official policy or position of NKS or any of its employees.



This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 19:22:33 EDT