mjhjmtu

I have a file with over 10,000 rows:

head samples 
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192170/type/READ_SET_FASTQ/filename/HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R1.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192170/type/READ_SET_FASTQ_PE/filename/HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R2.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192171/type/READ_SET_FASTQ/filename/HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R1.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192171/type/READ_SET_FASTQ_PE/filename/HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R2.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192168/type/READ_SET_FASTQ/filename/HI.2613.007.Index_18.ED17_MO1A26-7_353_R1.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192168/type/READ_SET_FASTQ_PE/filename/HI.2613.007.Index_18.ED17_MO1A26-7_353_R2.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192169/type/READ_SET_FASTQ/filename/HI.2613.007.Index_14.ED14_IA2A35-2_310_R1.fastq.gz.md5

I want to print only part of each line that starts with "HI.*"

This is my desired output:

HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R1.fastq.gz
HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R2.fastq.gz
HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R1.fastq.gz
HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R2.fastq.gz
HI.2613.007.Index_18.ED17_MO1A26-7_353_R1.fastq.gz
HI.2613.007.Index_18.ED17_MO1A26-7_353_R2.fastq.gz

Using awk

awk -F'/' '$NF ~ /^HI./ print $NF ' infile

to remove the .md5 suffix, you could do:

awk -F'(/|.md5)' '$(NF-1) ~ /^HI./ print $(NF-1) ' infile

• in awk, the $0 is referring to the whole line/record and $1, $2, $3, ... are referring to the first, second, third, ... respectively; and $NF referring to the last field and accordingly the $(NF-1) is the second last field.




• the tild ~ operator in awk treat the right-hand operator as (extended) regular-expression match against the left-hand operand as string string ~ /regular-expression/

The sed solution:

sed 's:.*/([^/]*).md5$:1: ; /^HI./!d' infile

• this /([^/]*).md5 matches last slash followed by anything but not a slash that ends with .md5. We take ([^/]*) (everything between last slash and .md5 as a group match and print just that in replacement part with its back-reference 1.




• this /^HI./!d deletes the lines which doesn't start with HI. from the result of previous sed command.




• we used different sed delimiter : since we have special / character in the input.

Try this,

awk -F '/' '$NF ~ /^HI/ print substr($NF, 1, length($NF)-4)' file.txt

• prints the last field if last field starts with HI
  


• excludes the last 4 charecters .md5
  

Output

HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R1.fastq.gz
HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R2.fastq.gz
HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R1.fastq.gz
HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R2.fastq.gz
HI.2613.007.Index_18.ED17_MO1A26-7_353_R1.fastq.gz
HI.2613.007.Index_18.ED17_MO1A26-7_353_R2.fastq.gz
HI.2613.007.Index_14.ED14_IA2A35-2_310_R1.fastq.gz

awk -F"filename/" 'gsub (".md5","");print $2'