Delete Substring from JSON file (malformed)
Clash Royale CLAN TAG#URR8PPP
up vote
2
down vote
favorite
I am trying to delete different values from a JSON file. It's not pretty and jq returns 1 compile error. I am searching for a solution from the command-line (awk/sed).
Here is my sample data from my json-file (similiar to docker-compose):
aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaa
bbbb"PROTO 9080/tcp 9090/tcp"bbbbb
cccc"TEST PROTO 80 8080"ccccc
All entries can be found multiple times or also within one line.
After using the replace it should look like:
aaaaaaaa
bbbbbbbbb
cccc"TEST"ccccc
I tried sed and (awk -F["]) but could not get some useful results.
How can this be done ?
Update: I made the sample clearer..
There are 3 different types:...:
1: Delete from "MyPorts" to }},
2. Delete "Proto to "
3. Delete from " PROTO" till one character before ", only if left from Proto is a space
Update 2: How can this be done if the sample data is also in one line, like this:
aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaabbbb"PROTO 9080/tcp 9090/tcp"bbbbbcccc"TEST PROTO 80 8080"ccccc
After using the replace it should look like:
aaaaaaaabbbbbbbbbcccc"TEST"ccccc
text-processing awk sed json
add a comment |Â
up vote
2
down vote
favorite
I am trying to delete different values from a JSON file. It's not pretty and jq returns 1 compile error. I am searching for a solution from the command-line (awk/sed).
Here is my sample data from my json-file (similiar to docker-compose):
aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaa
bbbb"PROTO 9080/tcp 9090/tcp"bbbbb
cccc"TEST PROTO 80 8080"ccccc
All entries can be found multiple times or also within one line.
After using the replace it should look like:
aaaaaaaa
bbbbbbbbb
cccc"TEST"ccccc
I tried sed and (awk -F["]) but could not get some useful results.
How can this be done ?
Update: I made the sample clearer..
There are 3 different types:...:
1: Delete from "MyPorts" to }},
2. Delete "Proto to "
3. Delete from " PROTO" till one character before ", only if left from Proto is a space
Update 2: How can this be done if the sample data is also in one line, like this:
aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaabbbb"PROTO 9080/tcp 9090/tcp"bbbbbcccc"TEST PROTO 80 8080"ccccc
After using the replace it should look like:
aaaaaaaabbbbbbbbbcccc"TEST"ccccc
text-processing awk sed json
2
this lineccccPROTO 80 8080"ccccc
looks strange. How diverse could be your lines? How this lineAaaA"aaa:1"cc"aaa"bbb""aaA
should be treated?
â RomanPerekhrest
Jun 14 at 18:02
I updated the sample to make it clearer.. There are 3 different types:...: 1: Delete from "MyPorts" to }}, 2. Delete "Proto to " . 3 Delete from " PROTO" till ", only if left from Proto is a space
â biohell
Jun 14 at 19:59
add a comment |Â
up vote
2
down vote
favorite
up vote
2
down vote
favorite
I am trying to delete different values from a JSON file. It's not pretty and jq returns 1 compile error. I am searching for a solution from the command-line (awk/sed).
Here is my sample data from my json-file (similiar to docker-compose):
aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaa
bbbb"PROTO 9080/tcp 9090/tcp"bbbbb
cccc"TEST PROTO 80 8080"ccccc
All entries can be found multiple times or also within one line.
After using the replace it should look like:
aaaaaaaa
bbbbbbbbb
cccc"TEST"ccccc
I tried sed and (awk -F["]) but could not get some useful results.
How can this be done ?
Update: I made the sample clearer..
There are 3 different types:...:
1: Delete from "MyPorts" to }},
2. Delete "Proto to "
3. Delete from " PROTO" till one character before ", only if left from Proto is a space
Update 2: How can this be done if the sample data is also in one line, like this:
aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaabbbb"PROTO 9080/tcp 9090/tcp"bbbbbcccc"TEST PROTO 80 8080"ccccc
After using the replace it should look like:
aaaaaaaabbbbbbbbbcccc"TEST"ccccc
text-processing awk sed json
I am trying to delete different values from a JSON file. It's not pretty and jq returns 1 compile error. I am searching for a solution from the command-line (awk/sed).
Here is my sample data from my json-file (similiar to docker-compose):
aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaa
bbbb"PROTO 9080/tcp 9090/tcp"bbbbb
cccc"TEST PROTO 80 8080"ccccc
All entries can be found multiple times or also within one line.
After using the replace it should look like:
aaaaaaaa
bbbbbbbbb
cccc"TEST"ccccc
I tried sed and (awk -F["]) but could not get some useful results.
How can this be done ?
Update: I made the sample clearer..
There are 3 different types:...:
1: Delete from "MyPorts" to }},
2. Delete "Proto to "
3. Delete from " PROTO" till one character before ", only if left from Proto is a space
Update 2: How can this be done if the sample data is also in one line, like this:
aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaabbbb"PROTO 9080/tcp 9090/tcp"bbbbbcccc"TEST PROTO 80 8080"ccccc
After using the replace it should look like:
aaaaaaaabbbbbbbbbcccc"TEST"ccccc
text-processing awk sed json
text-processing awk sed json
edited Aug 27 at 0:06
Jeff Schaller
32.7k849110
32.7k849110
asked Jun 14 at 17:56
biohell
133
133
2
this lineccccPROTO 80 8080"ccccc
looks strange. How diverse could be your lines? How this lineAaaA"aaa:1"cc"aaa"bbb""aaA
should be treated?
â RomanPerekhrest
Jun 14 at 18:02
I updated the sample to make it clearer.. There are 3 different types:...: 1: Delete from "MyPorts" to }}, 2. Delete "Proto to " . 3 Delete from " PROTO" till ", only if left from Proto is a space
â biohell
Jun 14 at 19:59
add a comment |Â
2
this lineccccPROTO 80 8080"ccccc
looks strange. How diverse could be your lines? How this lineAaaA"aaa:1"cc"aaa"bbb""aaA
should be treated?
â RomanPerekhrest
Jun 14 at 18:02
I updated the sample to make it clearer.. There are 3 different types:...: 1: Delete from "MyPorts" to }}, 2. Delete "Proto to " . 3 Delete from " PROTO" till ", only if left from Proto is a space
â biohell
Jun 14 at 19:59
2
2
this line
ccccPROTO 80 8080"ccccc
looks strange. How diverse could be your lines? How this line AaaA"aaa:1"cc"aaa"bbb""aaA
should be treated?â RomanPerekhrest
Jun 14 at 18:02
this line
ccccPROTO 80 8080"ccccc
looks strange. How diverse could be your lines? How this line AaaA"aaa:1"cc"aaa"bbb""aaA
should be treated?â RomanPerekhrest
Jun 14 at 18:02
I updated the sample to make it clearer.. There are 3 different types:...: 1: Delete from "MyPorts" to }}, 2. Delete "Proto to " . 3 Delete from " PROTO" till ", only if left from Proto is a space
â biohell
Jun 14 at 19:59
I updated the sample to make it clearer.. There are 3 different types:...: 1: Delete from "MyPorts" to }}, 2. Delete "Proto to " . 3 Delete from " PROTO" till ", only if left from Proto is a space
â biohell
Jun 14 at 19:59
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
sed
approach:
sed -E 's~("MyPorts.*},|"PROTO[^"]+"| +PROTO[^"]+)~~' file
The output:
aaaaaaaa
bbbbbbbbb
cccc"TEST"ccccc
Bonus approach for one-line string input:
$ s='aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaabbbb"PROTO 9080/tcp 9090/tcp"bbbbbcccc"TES^CPROTO 80 8080"ccccc'
$ sed -En 's~("MyPorts.+},|"PROTO[^"]+"| +PROTO[^"]+)~~gp' <<<"$s"
aaaaaaaabbbbbbbbbcccc"TEST"ccccc
Thank you very much. Works like a charm !!! Great and Short Solution.. Cannot mark this a solution due less reputation, Thanks RomanPerekhrest found it.
â biohell
Jun 14 at 20:28
Great job at interpreting the OP's request! @biohell can accept the answer, no minimum reputation is required.
â simlev
Jun 14 at 20:58
@RomanPerekhrest: if the sample data is in one line,your call doesn't work.. Can you chnage your call to reflect this ?
â biohell
Jun 15 at 9:40
@biohell, all additional cases should be described in the question (with respective input fragment)
â RomanPerekhrest
Jun 15 at 9:44
@@RomanPerekhrest: i updated the question.
â biohell
Jun 15 at 10:05
 |Â
show 2 more comments
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
sed
approach:
sed -E 's~("MyPorts.*},|"PROTO[^"]+"| +PROTO[^"]+)~~' file
The output:
aaaaaaaa
bbbbbbbbb
cccc"TEST"ccccc
Bonus approach for one-line string input:
$ s='aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaabbbb"PROTO 9080/tcp 9090/tcp"bbbbbcccc"TES^CPROTO 80 8080"ccccc'
$ sed -En 's~("MyPorts.+},|"PROTO[^"]+"| +PROTO[^"]+)~~gp' <<<"$s"
aaaaaaaabbbbbbbbbcccc"TEST"ccccc
Thank you very much. Works like a charm !!! Great and Short Solution.. Cannot mark this a solution due less reputation, Thanks RomanPerekhrest found it.
â biohell
Jun 14 at 20:28
Great job at interpreting the OP's request! @biohell can accept the answer, no minimum reputation is required.
â simlev
Jun 14 at 20:58
@RomanPerekhrest: if the sample data is in one line,your call doesn't work.. Can you chnage your call to reflect this ?
â biohell
Jun 15 at 9:40
@biohell, all additional cases should be described in the question (with respective input fragment)
â RomanPerekhrest
Jun 15 at 9:44
@@RomanPerekhrest: i updated the question.
â biohell
Jun 15 at 10:05
 |Â
show 2 more comments
up vote
1
down vote
accepted
sed
approach:
sed -E 's~("MyPorts.*},|"PROTO[^"]+"| +PROTO[^"]+)~~' file
The output:
aaaaaaaa
bbbbbbbbb
cccc"TEST"ccccc
Bonus approach for one-line string input:
$ s='aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaabbbb"PROTO 9080/tcp 9090/tcp"bbbbbcccc"TES^CPROTO 80 8080"ccccc'
$ sed -En 's~("MyPorts.+},|"PROTO[^"]+"| +PROTO[^"]+)~~gp' <<<"$s"
aaaaaaaabbbbbbbbbcccc"TEST"ccccc
Thank you very much. Works like a charm !!! Great and Short Solution.. Cannot mark this a solution due less reputation, Thanks RomanPerekhrest found it.
â biohell
Jun 14 at 20:28
Great job at interpreting the OP's request! @biohell can accept the answer, no minimum reputation is required.
â simlev
Jun 14 at 20:58
@RomanPerekhrest: if the sample data is in one line,your call doesn't work.. Can you chnage your call to reflect this ?
â biohell
Jun 15 at 9:40
@biohell, all additional cases should be described in the question (with respective input fragment)
â RomanPerekhrest
Jun 15 at 9:44
@@RomanPerekhrest: i updated the question.
â biohell
Jun 15 at 10:05
 |Â
show 2 more comments
up vote
1
down vote
accepted
up vote
1
down vote
accepted
sed
approach:
sed -E 's~("MyPorts.*},|"PROTO[^"]+"| +PROTO[^"]+)~~' file
The output:
aaaaaaaa
bbbbbbbbb
cccc"TEST"ccccc
Bonus approach for one-line string input:
$ s='aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaabbbb"PROTO 9080/tcp 9090/tcp"bbbbbcccc"TES^CPROTO 80 8080"ccccc'
$ sed -En 's~("MyPorts.+},|"PROTO[^"]+"| +PROTO[^"]+)~~gp' <<<"$s"
aaaaaaaabbbbbbbbbcccc"TEST"ccccc
sed
approach:
sed -E 's~("MyPorts.*},|"PROTO[^"]+"| +PROTO[^"]+)~~' file
The output:
aaaaaaaa
bbbbbbbbb
cccc"TEST"ccccc
Bonus approach for one-line string input:
$ s='aaaa"MyPorts":"80/tcp":,"8080/tcp":,"9080/tcp":,"9090/tcp":,aaaabbbb"PROTO 9080/tcp 9090/tcp"bbbbbcccc"TES^CPROTO 80 8080"ccccc'
$ sed -En 's~("MyPorts.+},|"PROTO[^"]+"| +PROTO[^"]+)~~gp' <<<"$s"
aaaaaaaabbbbbbbbbcccc"TEST"ccccc
edited Jun 15 at 10:11
answered Jun 14 at 20:17
RomanPerekhrest
22.5k12145
22.5k12145
Thank you very much. Works like a charm !!! Great and Short Solution.. Cannot mark this a solution due less reputation, Thanks RomanPerekhrest found it.
â biohell
Jun 14 at 20:28
Great job at interpreting the OP's request! @biohell can accept the answer, no minimum reputation is required.
â simlev
Jun 14 at 20:58
@RomanPerekhrest: if the sample data is in one line,your call doesn't work.. Can you chnage your call to reflect this ?
â biohell
Jun 15 at 9:40
@biohell, all additional cases should be described in the question (with respective input fragment)
â RomanPerekhrest
Jun 15 at 9:44
@@RomanPerekhrest: i updated the question.
â biohell
Jun 15 at 10:05
 |Â
show 2 more comments
Thank you very much. Works like a charm !!! Great and Short Solution.. Cannot mark this a solution due less reputation, Thanks RomanPerekhrest found it.
â biohell
Jun 14 at 20:28
Great job at interpreting the OP's request! @biohell can accept the answer, no minimum reputation is required.
â simlev
Jun 14 at 20:58
@RomanPerekhrest: if the sample data is in one line,your call doesn't work.. Can you chnage your call to reflect this ?
â biohell
Jun 15 at 9:40
@biohell, all additional cases should be described in the question (with respective input fragment)
â RomanPerekhrest
Jun 15 at 9:44
@@RomanPerekhrest: i updated the question.
â biohell
Jun 15 at 10:05
Thank you very much. Works like a charm !!! Great and Short Solution.. Cannot mark this a solution due less reputation, Thanks RomanPerekhrest found it.
â biohell
Jun 14 at 20:28
Thank you very much. Works like a charm !!! Great and Short Solution.. Cannot mark this a solution due less reputation, Thanks RomanPerekhrest found it.
â biohell
Jun 14 at 20:28
Great job at interpreting the OP's request! @biohell can accept the answer, no minimum reputation is required.
â simlev
Jun 14 at 20:58
Great job at interpreting the OP's request! @biohell can accept the answer, no minimum reputation is required.
â simlev
Jun 14 at 20:58
@RomanPerekhrest: if the sample data is in one line,your call doesn't work.. Can you chnage your call to reflect this ?
â biohell
Jun 15 at 9:40
@RomanPerekhrest: if the sample data is in one line,your call doesn't work.. Can you chnage your call to reflect this ?
â biohell
Jun 15 at 9:40
@biohell, all additional cases should be described in the question (with respective input fragment)
â RomanPerekhrest
Jun 15 at 9:44
@biohell, all additional cases should be described in the question (with respective input fragment)
â RomanPerekhrest
Jun 15 at 9:44
@@RomanPerekhrest: i updated the question.
â biohell
Jun 15 at 10:05
@@RomanPerekhrest: i updated the question.
â biohell
Jun 15 at 10:05
 |Â
show 2 more comments
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f449870%2fdelete-substring-from-json-file-malformed%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
2
this line
ccccPROTO 80 8080"ccccc
looks strange. How diverse could be your lines? How this lineAaaA"aaa:1"cc"aaa"bbb""aaA
should be treated?â RomanPerekhrest
Jun 14 at 18:02
I updated the sample to make it clearer.. There are 3 different types:...: 1: Delete from "MyPorts" to }}, 2. Delete "Proto to " . 3 Delete from " PROTO" till ", only if left from Proto is a space
â biohell
Jun 14 at 19:59