How to insert a line in a file after certain bytes

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

Let's say I have a large file (several gigs) with n lines in it. I would like to add/insert a line after k bytes offset, from the beginning of the file, then what would be the fastest way to achieve that?

edited Mar 10 at 12:04

Jeff Schaller♦

44.7k1163145

asked Mar 10 at 5:51

khan

1112

what did you try until now?

– yael
Mar 10 at 5:53

please let us know why you want to do it?

– yael
Mar 10 at 5:54

I was thinking of trying something like head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file; but not sure if its the best way to do this simple task..

– khan
Mar 10 at 5:56

1

@yael it doesn't really matter why he wants to do it.

– RonJohn
Mar 10 at 6:38

1

For Very Large Files, your standard Unix text tools might not be the best solution. It certainly can't hurt to try head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file especially if it's a one-time operation. It might have already completed by now.

– RonJohn
Mar 10 at 7:32

|
show 7 more comments

edited Mar 10 at 12:04

Jeff Schaller♦

44.7k1163145

asked Mar 10 at 5:51

khan

1112

what did you try until now?

– yael
Mar 10 at 5:53

please let us know why you want to do it?

– yael
Mar 10 at 5:54

I was thinking of trying something like head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file; but not sure if its the best way to do this simple task..

– khan
Mar 10 at 5:56

1

@yael it doesn't really matter why he wants to do it.

– RonJohn
Mar 10 at 6:38

1

For Very Large Files, your standard Unix text tools might not be the best solution. It certainly can't hurt to try head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file especially if it's a one-time operation. It might have already completed by now.

– RonJohn
Mar 10 at 7:32

|
show 7 more comments

edited Mar 10 at 12:04

Jeff Schaller♦

44.7k1163145

asked Mar 10 at 5:51

khan

1112

linux files cut tail

edited Mar 10 at 12:04

Jeff Schaller♦

44.7k1163145

asked Mar 10 at 5:51

khan

1112

edited Mar 10 at 12:04

Jeff Schaller♦

44.7k1163145

asked Mar 10 at 5:51

khan

1112

edited Mar 10 at 12:04

Jeff Schaller♦

44.7k1163145

edited Mar 10 at 12:04

Jeff Schaller♦

44.7k1163145

edited Mar 10 at 12:04

Jeff Schaller♦

44.7k1163145

asked Mar 10 at 5:51

khan

1112

asked Mar 10 at 5:51

khan

1112

asked Mar 10 at 5:51

khan

1112

what did you try until now?

– yael
Mar 10 at 5:53

please let us know why you want to do it?

– yael
Mar 10 at 5:54

I was thinking of trying something like head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file; but not sure if its the best way to do this simple task..

– khan
Mar 10 at 5:56

1

@yael it doesn't really matter why he wants to do it.

– RonJohn
Mar 10 at 6:38

1

For Very Large Files, your standard Unix text tools might not be the best solution. It certainly can't hurt to try head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file especially if it's a one-time operation. It might have already completed by now.

– RonJohn
Mar 10 at 7:32

|
show 7 more comments

what did you try until now?

– yael
Mar 10 at 5:53

please let us know why you want to do it?

– yael
Mar 10 at 5:54

I was thinking of trying something like head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file; but not sure if its the best way to do this simple task..

– khan
Mar 10 at 5:56

1

@yael it doesn't really matter why he wants to do it.

– RonJohn
Mar 10 at 6:38

1

For Very Large Files, your standard Unix text tools might not be the best solution. It certainly can't hurt to try head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file especially if it's a one-time operation. It might have already completed by now.

– RonJohn
Mar 10 at 7:32

what did you try until now?

– yael
Mar 10 at 5:53

please let us know why you want to do it?

– yael
Mar 10 at 5:54

I was thinking of trying something like head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file; but not sure if its the best way to do this simple task..

– khan
Mar 10 at 5:56

@yael it doesn't really matter why he wants to do it.

– RonJohn
Mar 10 at 6:38

For Very Large Files, your standard Unix text tools might not be the best solution. It certainly can't hurt to try head -c k file > temp.log; echo 'some datan' >> temp.log; tail -c +(k+1) file > temp.log; mv temp.log file especially if it's a one-time operation. It might have already completed by now.

– RonJohn
Mar 10 at 7:32

|
show 7 more comments

2 Answers
2

active

oldest

votes

Here's a Python solution:

#!/usr/bin/env python3
# -*- encoding: utf-8 -*-
"""split_bytes.py"""

import os
import sys

stdout = os.fdopen(sys.stdout.fileno(), 'wb')

path_to_file = sys.argv[1]
width_in_bytes = int(sys.argv[2])

with open(path_to_file, "rb") as f:
 byte = f.read(1)
 while byte:
 for i in range(width_in_bytes):
 stdout.write(byte)
 byte = f.read(1)
 stdout.write(b"n")

You could execute it like this:

python split_bytes.py path/to/file offset > new_file

As a test, I generated a 1GB file of random data:

dd if=/dev/urandom of=data.bin bs=64M count=16 iflag=fullblock

Then ran the script on that file:

python split_lines.py data.bin 10 > split-data.bin

edited Mar 11 at 3:02

answered Mar 11 at 1:40

igal

6,1411638

This looks like a promising approach..let me try some tests.

– khan
Mar 11 at 3:05

add a comment |

a bash only solution :

use split command :

split --lines=2 --suffix-length=6 /etc/passwd /tmp/split.passwd.part

reassemble the file into one new

(
 for F in /tmp/split.passwd.part* ; 
 do 
 cat $F ; 
 echo ; 
 done
) > /tmp/passwd_emptyline_evrey_2

answered Mar 11 at 3:51

EchoMike444

1,0506

add a comment |

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

draft saved

draft discarded

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f505421%2fhow-to-insert-a-line-in-a-file-after-certain-bytes%23new-answer', 'question_page');

);

Post as a guest

Name

Required, but never shown

2 Answers
2

active

oldest

votes

2 Answers
2

active

oldest

votes

Here's a Python solution:

#!/usr/bin/env python3
# -*- encoding: utf-8 -*-
"""split_bytes.py"""

import os
import sys

stdout = os.fdopen(sys.stdout.fileno(), 'wb')

path_to_file = sys.argv[1]
width_in_bytes = int(sys.argv[2])

with open(path_to_file, "rb") as f:
 byte = f.read(1)
 while byte:
 for i in range(width_in_bytes):
 stdout.write(byte)
 byte = f.read(1)
 stdout.write(b"n")

You could execute it like this:

python split_bytes.py path/to/file offset > new_file

As a test, I generated a 1GB file of random data:

dd if=/dev/urandom of=data.bin bs=64M count=16 iflag=fullblock

Then ran the script on that file:

python split_lines.py data.bin 10 > split-data.bin

edited Mar 11 at 3:02

answered Mar 11 at 1:40

igal

6,1411638

This looks like a promising approach..let me try some tests.

– khan
Mar 11 at 3:05

add a comment |

Here's a Python solution:

#!/usr/bin/env python3
# -*- encoding: utf-8 -*-
"""split_bytes.py"""

import os
import sys

stdout = os.fdopen(sys.stdout.fileno(), 'wb')

path_to_file = sys.argv[1]
width_in_bytes = int(sys.argv[2])

with open(path_to_file, "rb") as f:
 byte = f.read(1)
 while byte:
 for i in range(width_in_bytes):
 stdout.write(byte)
 byte = f.read(1)
 stdout.write(b"n")

You could execute it like this:

python split_bytes.py path/to/file offset > new_file

As a test, I generated a 1GB file of random data:

dd if=/dev/urandom of=data.bin bs=64M count=16 iflag=fullblock

Then ran the script on that file:

python split_lines.py data.bin 10 > split-data.bin

edited Mar 11 at 3:02

answered Mar 11 at 1:40

igal

6,1411638

This looks like a promising approach..let me try some tests.

– khan
Mar 11 at 3:05

add a comment |

Here's a Python solution:

#!/usr/bin/env python3
# -*- encoding: utf-8 -*-
"""split_bytes.py"""

import os
import sys

stdout = os.fdopen(sys.stdout.fileno(), 'wb')

path_to_file = sys.argv[1]
width_in_bytes = int(sys.argv[2])

with open(path_to_file, "rb") as f:
 byte = f.read(1)
 while byte:
 for i in range(width_in_bytes):
 stdout.write(byte)
 byte = f.read(1)
 stdout.write(b"n")

You could execute it like this:

python split_bytes.py path/to/file offset > new_file

As a test, I generated a 1GB file of random data:

dd if=/dev/urandom of=data.bin bs=64M count=16 iflag=fullblock

Then ran the script on that file:

python split_lines.py data.bin 10 > split-data.bin

edited Mar 11 at 3:02

answered Mar 11 at 1:40

igal

6,1411638

Here's a Python solution:

#!/usr/bin/env python3
# -*- encoding: utf-8 -*-
"""split_bytes.py"""

import os
import sys

stdout = os.fdopen(sys.stdout.fileno(), 'wb')

path_to_file = sys.argv[1]
width_in_bytes = int(sys.argv[2])

with open(path_to_file, "rb") as f:
 byte = f.read(1)
 while byte:
 for i in range(width_in_bytes):
 stdout.write(byte)
 byte = f.read(1)
 stdout.write(b"n")

You could execute it like this:

python split_bytes.py path/to/file offset > new_file

As a test, I generated a 1GB file of random data:

dd if=/dev/urandom of=data.bin bs=64M count=16 iflag=fullblock

Then ran the script on that file:

python split_lines.py data.bin 10 > split-data.bin

edited Mar 11 at 3:02

answered Mar 11 at 1:40

igal

6,1411638

edited Mar 11 at 3:02

answered Mar 11 at 1:40

igal

6,1411638

answered Mar 11 at 1:40

igal

6,1411638

answered Mar 11 at 1:40

igal

6,1411638

This looks like a promising approach..let me try some tests.

– khan
Mar 11 at 3:05

add a comment |

This looks like a promising approach..let me try some tests.

– khan
Mar 11 at 3:05

This looks like a promising approach..let me try some tests.

– khan
Mar 11 at 3:05

add a comment |

a bash only solution :

use split command :

split --lines=2 --suffix-length=6 /etc/passwd /tmp/split.passwd.part

reassemble the file into one new

(
 for F in /tmp/split.passwd.part* ; 
 do 
 cat $F ; 
 echo ; 
 done
) > /tmp/passwd_emptyline_evrey_2

answered Mar 11 at 3:51

EchoMike444

1,0506

add a comment |

a bash only solution :

use split command :

split --lines=2 --suffix-length=6 /etc/passwd /tmp/split.passwd.part

reassemble the file into one new

(
 for F in /tmp/split.passwd.part* ; 
 do 
 cat $F ; 
 echo ; 
 done
) > /tmp/passwd_emptyline_evrey_2

answered Mar 11 at 3:51

EchoMike444

1,0506

add a comment |

a bash only solution :

use split command :

split --lines=2 --suffix-length=6 /etc/passwd /tmp/split.passwd.part

reassemble the file into one new

(
 for F in /tmp/split.passwd.part* ; 
 do 
 cat $F ; 
 echo ; 
 done
) > /tmp/passwd_emptyline_evrey_2

answered Mar 11 at 3:51

EchoMike444

1,0506

a bash only solution :

use split command :

split --lines=2 --suffix-length=6 /etc/passwd /tmp/split.passwd.part

reassemble the file into one new

(
 for F in /tmp/split.passwd.part* ; 
 do 
 cat $F ; 
 echo ; 
 done
) > /tmp/passwd_emptyline_evrey_2

answered Mar 11 at 3:51

EchoMike444

1,0506

answered Mar 11 at 3:51

EchoMike444

1,0506

answered Mar 11 at 3:51

EchoMike444

1,0506

answered Mar 11 at 3:51

EchoMike444

1,0506

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Unix & Linux Stack Exchange!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

搜尋此網誌

mjhjmtu