So I have a textfile of which I want to remove everything to the first colon (including the colon). So for example if this is the input 0000007ba9ec6950086ce79a8f3a389db4235830:9515rfsvk 000000da2a12da3fbe01a95bddb8ee183c62b94d:letmein2x 000000edf3179a1cf4c354471a897ab7f420bd52:heychudi:rbhai 000000f636f0d7cbc963a62f3a1bc87c9c985a04:cornetti 0000010a15f5b9315ef8e113f139fa413d1f2eb2:3648067PY128  Then this should be the output 9515rfsvk letmein2x heychudi:rbhai cornetti 3648067PY128  Note that the second colon in line 3 remain, only from the start of each line to (including) the first column should be removed. Is there a quick way to do this with grep or awk?
93119
I am trying to delete certain text that appears before / on every line.   I have something like:  testing.db.com/7fad416d-f2b3-4259-b98d-2449957a3123 testing.db.com/8a8589bf-49e3-4cd7-af15-6753067355c6   and I just want to end up with:   7fad416d-f2b3-4259-b98d-2449957a3123 8a8589bf-49e3-4cd7-af15-6753067355c6    Can anyone help me with a regular expression? Everything I have found is deleting after /, not before.
4618
I have a file containing two columns and 10 million rows. The first column contains many repeated values, but there is a distinct value in column 2. I want to remove the repeated rows and want to keep only one using awk. Note: the file is sorted with values in column 1. For example:    1.123 -4.0 2.234 -3.5 2.234 -3.1 2.234 -2.0 4.432 0.0 5.123 +0.2 8.654 +0.5 8.654 +0.8 8.654 +0.9 . . . .   Expected output  1.123 -4.0 2.234 -3.5 4.432 0.0 5.123 +0.2 8.654 +0.5 . . . .
6565
I have two sample files like this:  $ cat file1 abc,sachin cat,kumar   $ cat file2 xyz pressure  $ cat file3  xyz,sachin pressure, kumar    I want the first column of file1 to be replaced with file2.  I tried doing something like this:  $ awk 'FNR==NR{a[NR]=$3;next}{$2=a[FNR]}1' file1 file2   I'm using Solaris 10 and it didn't appear to support this. Any other suggestions?
122203
I have a Bash script that writes data into a log file, then from that I take out stuff I want using the sed command. That gives me a file data.txt. What I want to know is how I go about deleting specific line inside that data.txt file.  For example:  123 456 789   I want to remove the 2nd line, containing 456 so that I only have  123  789   I tried with sed '2d' data.txt but it doesn't work.  I use this command to create data.txt from log.log:  sed -nE '1s/.{1}(.{2}).*/\1/p;' log.log &gt;&gt; data.txt
107278
I have a text file named abd shown below.   48878 128.206.6.136 34782 128.206.6.137 12817 23.234.22.106   I want to extract only IP address from the text and store it in a variable and use for other purpose.  I have tried this.  for line in `cat abd` do  ip=`grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' $line`  echo $ip  done   I am getting an error as follows  grep: 34782: No such file or directory  grep: 128.206.6.137: No such file or directory  grep: 12817: No such file or directory  grep: 23.234.22.106: No such file or directory   I don't know what is going wrong here.  Any help would be appreciated.
36910
To be precise  Some text begin Some text goes here. end Some more text   and I want to extract entire block that starts from "begin" till "end".  with awk we can do like awk '/begin/,/end/' text.  How to do with grep?
44760
I need to repeatedly remove the first line from a huge text file using a bash script.  Right now I am using sed -i -e "1d" $FILE - but it takes around a minute to do the deletion.  Is there a more efficient way to accomplish this?
19174
I need to remove every line that has a value of 2 or less in the 8th field (column).   My data looks like this:  12-31   Airport 189 379 41  49.70946503 -124.91377258   2   2880    30.8 01-01   AlberniElementary   165 331 16  49.26100922 -124.80662537   4   5760    26.1 01-09   BamfieldMarine  161 323 23  48.83490372 -125.13572693   2   2875    27.4 01-10   BamfieldMarine  161 323 23  48.83490372 -125.13572693   3   3068    38.6   I understand that using awk I can strip off the values desired and print them to another file, and I understand that sed would edit the current file. In either case, I need to retain the original file.   Note: Please provide thorough explanations with your solutions. It is not suffice to just write the command, please annotate suggestions.   Further note: The data has a header line, so most likely solution will need to      awk 'FNR >1'   I suppose?
10395
I've got a bunch of files named with a the pattern 99 - DescriptiveName.txt and I'd like to remove the number from the front so I just have DescriptiveName.txt.  How can I do this? Can I do it from the command line or is there a utility that can do this?
94793
I have a text file like so  foo bar baz 1   a   alpha 2   b   beta 3   c   gamma   I can use awk to print certain columns, like 1 and 3, with {print $1, $3}, but I want to specify the columns to print by specifying the header of the column instead, something like {print $foo, $baz}. This is useful so I don't have to open the file and count the columns manually to see which column is which, and I don't have to update the script if the column number or order changes. Can I do this with awk (or another shell tool)?
68162
doesn't seem to work  The next example would delete everything between "ONE" and "TWO:"   #!/bin/sh sed ' /ONE/ { # append a line     N # search for TWO on the second line      /\n.*TWO/ { # found it - now edit making one line         s/ONE.*\n.*TWO/ONE TWO/     } }' file   i was expecting to delete everything between ONE and TWO.i tried with file where file contains.  [user5@mailserver ~]$ cat file ONE hello TWO   but the output come like   [user5@mailserver ~]$ ./test ONE hello TWO   I was expecting ONE TWO I am trying to make it correct but not able to do.
61366
I have a file containing lines:  india;austria;japan;chile china;US;nigeria;mexico;russia   I want to replace all the occurences of semicolon on each line with e.g. ;NEW;, but starting from the 2nd occurence only. The result should look like this:  india;austria;NEW;japan;NEW;chile china;US;NEW;nigeria;NEW;mexico;NEW;russia   I tried this with gsub, but it replaces all the occurences:  awk '/;/{gsub(/;/,";NEW;") }{print}'
38993
I am new to UNIX and am trying to learn basic level text processing in UNIX so maybe this question may seem very basic but I would really appreciate any guidance.  I have a text file as below  A 1 B 2 C 9 D 1 A 5 B 3 C 4 A 6 C 7 A 5 C 1   for which I was able to piece together a command with some help as below  cat | awk 'BEGIN{OFS=FS=" "}!(($1,$2)in c){c[$1,$2]=1;r[$1]=r[$1] OFS $2} END{for(i in r){print i r[i]}}'   inside a shell script to get it to the format below:  A 1 5 6 5 B 2 3 C 9 4 7 1 D 1   My shell script will also take an argument k  (=7 say)  In this case, my script should sample 1 element from each box (A,B,C,D) until it has a total of 7 elements sampled. This will be like follows: First pick 1 from A, then 2 from B, 9 from C, 1 from D, 5 from A, 3 from B, 4 from C and display  A 1 5 B 2 3 C 9 4 D 1   If instead I had k = 9 as my input, then my output should be  A 1 5 6 B 2 3 C 9 4 7 D 1
68545
A related question is .  I often have to edit a large file by removing a few lines from the middle of it. I know which lines I wish to remove and I typically do the following:  sed "linenum1,linenum2 d" input.txt &gt; input.temp   or in-line by adding the -i option. Since I know the line numbers, is there a command to avoid stream-editing and just remove the particular lines? input.txt can be as large as 50 GB.
95010
I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row    123,"ABC, DEV 23",345,534.202,NAME   I need to remove all the comma's that are not occuring within the double quotes by a ~ So the output should be like:    123~"ABC, DEV 23"~345~534.202~NAME   I have tried this, but it gives me reverse output:  awk -F '"' -v OFS='' '{ for (i=0; i&lt;= NF; ++i) gsub(",","~",$i) } 1' test.txt 123,ABC~ DEV 23,345,534.202,NAME
37118
I want to find patterns that are listed in one file and find them in other file. The second file has those patterns separated by commas.  for e.g. first file F1 has genes  ENSG00000187546 ENSG00000113492   ENSG00000166971   and second file F2 has those genes along with some more columns(five columns) which I need   region     gene           chromosome  start       end   intronic   ENSG00000135870 1   173921301   173921301 intergenic  ENSG00000166971(dist=56181),ENSG00000103494(dist=37091) 16 53594504    53594504 ncRNA_intronic  ENSG00000215231 5   5039185 5039185 intronic    ENSG00000157890 15  66353740    66353740   So the gene ENSG00000166971 which is present in the second file does not show up in grep because it has another gene with it,separated by comma.  My code is:  grep -f "F1.txt" "F2.txt" &gt;output.txt   I want those values even if one of them is present,and the associated data with it.Is there any way to do this?
81695
I have a file as below:  mime PI Name: ISHO SUCCESS RATE RT, Value: 95.663826 scr  PI Name: RRC Access Failures due to UU, Value: 0.13394141 prog PI Name: RRC Access Failures due to UU, Value: 0.16077702 sch PI Name: RRC Access Failures due to UU, Value: 0.11781933   I want to remove text PI till Value:. I tried   sed '/&lt;PI&gt;/,/&lt;\/Value:&gt;/d'    Any help?