removing “na” from a column and collapsing the row

Clash Royale CLAN TAG#URR8PPP
I would like to remove rows that have an "na"
file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
na
na
0.000
0.000
na
0.002
0.002
0.003
output
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
I was trying to do this in R but i couldn't collapse the rows, just remove the na
awk sed grep
add a comment |
I would like to remove rows that have an "na"
file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
na
na
0.000
0.000
na
0.002
0.002
0.003
output
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
I was trying to do this in R but i couldn't collapse the rows, just remove the na
awk sed grep
4
Withgrep -v na. Removes all lines containing the stringna.The-vis the short form for--invert-match.
– Stefan Hamcke
Dec 29 '18 at 21:22
If you decide you do want to do it in R, it would be something likedata = read.table("data.txt", na.strings = "na")thendata = na.omit(data)I think
– steeldriver
Dec 29 '18 at 21:51
add a comment |
I would like to remove rows that have an "na"
file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
na
na
0.000
0.000
na
0.002
0.002
0.003
output
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
I was trying to do this in R but i couldn't collapse the rows, just remove the na
awk sed grep
I would like to remove rows that have an "na"
file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
na
na
0.000
0.000
na
0.002
0.002
0.003
output
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
I was trying to do this in R but i couldn't collapse the rows, just remove the na
awk sed grep
awk sed grep
edited Dec 29 '18 at 23:42
Rui F Ribeiro
39.4k1479131
39.4k1479131
asked Dec 29 '18 at 21:21
Age87Age87
1477
1477
4
Withgrep -v na. Removes all lines containing the stringna.The-vis the short form for--invert-match.
– Stefan Hamcke
Dec 29 '18 at 21:22
If you decide you do want to do it in R, it would be something likedata = read.table("data.txt", na.strings = "na")thendata = na.omit(data)I think
– steeldriver
Dec 29 '18 at 21:51
add a comment |
4
Withgrep -v na. Removes all lines containing the stringna.The-vis the short form for--invert-match.
– Stefan Hamcke
Dec 29 '18 at 21:22
If you decide you do want to do it in R, it would be something likedata = read.table("data.txt", na.strings = "na")thendata = na.omit(data)I think
– steeldriver
Dec 29 '18 at 21:51
4
4
With
grep -v na. Removes all lines containing the string na.The -v is the short form for --invert-match.– Stefan Hamcke
Dec 29 '18 at 21:22
With
grep -v na. Removes all lines containing the string na.The -v is the short form for --invert-match.– Stefan Hamcke
Dec 29 '18 at 21:22
If you decide you do want to do it in R, it would be something like
data = read.table("data.txt", na.strings = "na") then data = na.omit(data) I think– steeldriver
Dec 29 '18 at 21:51
If you decide you do want to do it in R, it would be something like
data = read.table("data.txt", na.strings = "na") then data = na.omit(data) I think– steeldriver
Dec 29 '18 at 21:51
add a comment |
3 Answers
3
active
oldest
votes
With awk:
awk '!/na/' file > new_file
The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:
awk '!/na/ print' file > new_file
To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.
awk '!/na/ && !/^$/ print' file > new_file
With sed:
sed '/na/d' file > new_file
The "/d tells sed to delete the line containing the matched pattern "/na/"
To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed
sed -i '/na/d' file
Using nothing but the bash shell with bash builtin:
while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file
With Python:
with open('file', 'r') as rf:
with open('out', 'w') as of:
for line in rf:
if not "na" in line:
of.write(line)
As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.
$ cat file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
add a comment |
As simple as:
grep -v na file > new_file
You'd better use-xwithgrephere to only match across complete lines. I would probably usegrep -x -F -v nato be absolutely correct.
– Kusalananda
Jan 7 at 22:34
add a comment |
I tried with below 3 methods
1.using awk
command:awk '$0 !~/na/print $0' filename
2. using sed
sed -n '/na/!p' filename
3. using python
#!/usr/bin/python
import re
k=open('filename','r')
a=re.compile(r'[0-9]')
for i in k:
if re.search(a,i):
print i,
output
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491519%2fremoving-na-from-a-column-and-collapsing-the-row%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
With awk:
awk '!/na/' file > new_file
The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:
awk '!/na/ print' file > new_file
To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.
awk '!/na/ && !/^$/ print' file > new_file
With sed:
sed '/na/d' file > new_file
The "/d tells sed to delete the line containing the matched pattern "/na/"
To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed
sed -i '/na/d' file
Using nothing but the bash shell with bash builtin:
while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file
With Python:
with open('file', 'r') as rf:
with open('out', 'w') as of:
for line in rf:
if not "na" in line:
of.write(line)
As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.
$ cat file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
add a comment |
With awk:
awk '!/na/' file > new_file
The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:
awk '!/na/ print' file > new_file
To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.
awk '!/na/ && !/^$/ print' file > new_file
With sed:
sed '/na/d' file > new_file
The "/d tells sed to delete the line containing the matched pattern "/na/"
To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed
sed -i '/na/d' file
Using nothing but the bash shell with bash builtin:
while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file
With Python:
with open('file', 'r') as rf:
with open('out', 'w') as of:
for line in rf:
if not "na" in line:
of.write(line)
As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.
$ cat file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
add a comment |
With awk:
awk '!/na/' file > new_file
The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:
awk '!/na/ print' file > new_file
To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.
awk '!/na/ && !/^$/ print' file > new_file
With sed:
sed '/na/d' file > new_file
The "/d tells sed to delete the line containing the matched pattern "/na/"
To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed
sed -i '/na/d' file
Using nothing but the bash shell with bash builtin:
while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file
With Python:
with open('file', 'r') as rf:
with open('out', 'w') as of:
for line in rf:
if not "na" in line:
of.write(line)
As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.
$ cat file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
With awk:
awk '!/na/' file > new_file
The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:
awk '!/na/ print' file > new_file
To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.
awk '!/na/ && !/^$/ print' file > new_file
With sed:
sed '/na/d' file > new_file
The "/d tells sed to delete the line containing the matched pattern "/na/"
To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed
sed -i '/na/d' file
Using nothing but the bash shell with bash builtin:
while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file
With Python:
with open('file', 'r') as rf:
with open('out', 'w') as of:
for line in rf:
if not "na" in line:
of.write(line)
As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.
$ cat file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
edited Dec 30 '18 at 11:32
answered Dec 30 '18 at 10:41
Chris SmithChris Smith
564
564
add a comment |
add a comment |
As simple as:
grep -v na file > new_file
You'd better use-xwithgrephere to only match across complete lines. I would probably usegrep -x -F -v nato be absolutely correct.
– Kusalananda
Jan 7 at 22:34
add a comment |
As simple as:
grep -v na file > new_file
You'd better use-xwithgrephere to only match across complete lines. I would probably usegrep -x -F -v nato be absolutely correct.
– Kusalananda
Jan 7 at 22:34
add a comment |
As simple as:
grep -v na file > new_file
As simple as:
grep -v na file > new_file
answered Dec 30 '18 at 1:47
Emilio GalarragaEmilio Galarraga
51129
51129
You'd better use-xwithgrephere to only match across complete lines. I would probably usegrep -x -F -v nato be absolutely correct.
– Kusalananda
Jan 7 at 22:34
add a comment |
You'd better use-xwithgrephere to only match across complete lines. I would probably usegrep -x -F -v nato be absolutely correct.
– Kusalananda
Jan 7 at 22:34
You'd better use
-x with grep here to only match across complete lines. I would probably use grep -x -F -v na to be absolutely correct.– Kusalananda
Jan 7 at 22:34
You'd better use
-x with grep here to only match across complete lines. I would probably use grep -x -F -v na to be absolutely correct.– Kusalananda
Jan 7 at 22:34
add a comment |
I tried with below 3 methods
1.using awk
command:awk '$0 !~/na/print $0' filename
2. using sed
sed -n '/na/!p' filename
3. using python
#!/usr/bin/python
import re
k=open('filename','r')
a=re.compile(r'[0-9]')
for i in k:
if re.search(a,i):
print i,
output
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
add a comment |
I tried with below 3 methods
1.using awk
command:awk '$0 !~/na/print $0' filename
2. using sed
sed -n '/na/!p' filename
3. using python
#!/usr/bin/python
import re
k=open('filename','r')
a=re.compile(r'[0-9]')
for i in k:
if re.search(a,i):
print i,
output
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
add a comment |
I tried with below 3 methods
1.using awk
command:awk '$0 !~/na/print $0' filename
2. using sed
sed -n '/na/!p' filename
3. using python
#!/usr/bin/python
import re
k=open('filename','r')
a=re.compile(r'[0-9]')
for i in k:
if re.search(a,i):
print i,
output
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
I tried with below 3 methods
1.using awk
command:awk '$0 !~/na/print $0' filename
2. using sed
sed -n '/na/!p' filename
3. using python
#!/usr/bin/python
import re
k=open('filename','r')
a=re.compile(r'[0-9]')
for i in k:
if re.search(a,i):
print i,
output
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003
answered Dec 31 '18 at 5:27
Praveen Kumar BSPraveen Kumar BS
1,250138
1,250138
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491519%2fremoving-na-from-a-column-and-collapsing-the-row%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
4
With
grep -v na. Removes all lines containing the stringna.The-vis the short form for--invert-match.– Stefan Hamcke
Dec 29 '18 at 21:22
If you decide you do want to do it in R, it would be something like
data = read.table("data.txt", na.strings = "na")thendata = na.omit(data)I think– steeldriver
Dec 29 '18 at 21:51