I used wget within bash to download a bunch of files from a parameterized URL.  My URL file looked something like this: http://www.example.com/Home/Sales?lrsn=48726 http://www.example.com/Home/Sales?lrsn=48727 http://www.example.com/Home/Sales?lrsn=48728 ...and so on  I ran this wget command to fetch all the files into my directory: wget -i urlfile.txt  The resulting set of files-- when I do an ls in my directory-- has names like this: 'Sales?lrsn=48726' 'Sales?lrsn=48727' 'Sales?lrsn=48728'  My problem is that I cannot figure out how to use wildcards with those files.  E.g., if I type ls 'Sales*8', I would expect it to list 'Sales?lrsn=48728' (and any other matches).  Instead, I get &quot;No such file or directory&quot;. Questions:  What do I need to do to use wildcards with the files that my wget operation created? Simple example would be how to match using ls. How would one even create such files to reproduce this.  I.e., I tried touch 'text.txt' and touch &quot;'test.txt'&quot; and other variations, but I can't seem to make a file that lists as 'test.txt' (starting and ending single-quotes). Is there a simple way to provide a naming convention for downloads using the wget -i input mode? (I'm pretty sure the answer to this is &quot;No,&quot; but... figured I'd ask.)
36541
I just noticed that on one of my machines (running Debian Sid) whenever I type ls any file name with spaces has single quotes surrounding it. I immediately checked my aliases, only to find them intact. wyatt@debian630:~/testdir$ ls 'test 1.txt'  test1.txt wyatt@debian630:~/testdir$ alias alias ls='ls --color=auto' alias wget='wget --content-disposition' wyatt@debian630:~/testdir$   Another test, with files containing single quotes in their names (also answering a request by jimmij): wyatt@debian630:~/testdir$ ls 'test 1.txt'  test1.txt  'thishasasinglequotehere'\''.txt' wyatt@debian630:~/testdir$ touch &quot;'test 1.txt'&quot; wyatt@debian630:~/testdir$ ls ''\''test 1.txt'\'''  test1.txt 'test 1.txt'          'thishasasinglequotehere'\''.txt'   update with new coreutils-8.26 output (which is admittedly much less confusing, but still irritating to have by default). Thanks to Pádraig Brady for this printout: $ ls &quot;'test 1.txt'&quot;   test1.txt 'test 1.txt'    &quot;thishasasinglequotehere'.txt&quot;  $ ls -N 'test 1.txt'  test1.txt test 1.txt    thishasasinglequotehere'.txt  Why is this happening? How do I stop it properly? To be clear, I myself set ls to automatically color output. It just never put quotes around things before. I'm running bash and coreutils 8.25. Any way to fix this without a recompile? EDIT: Appears the coreutils developers  to break with the convention and make this the global default.  UPDATE - October 2017 - Debian Sid has re-enabled the shell escape quoting by default.  And at the bottom of the reply chain to the previous bug report, &quot;the change was intentional and will remain.&quot;  I thought this had already been settled, but apparently it was just reverted so that the &quot;stable&quot; Debian branch could keep its &quot;feature freeze&quot; while getting the other fixes, etc. from the newer version. So that's a shame (in my opinion). UPDATE: April 2019: Just found a  that was caused by this change to ls. When you're confusing developers and generating false bug reports, I think it might be time to re-evaluate your changes. Update: Android toybox ls is now doing something similar to this but with backslashes instead of quotes. Using the -q option makes spaces render as 'question mark characters' (I have not checked what they are, since they're obviously not spaces), so the only fix I have found so far without rooting the device in question is to add this to a script and source it when launching a shell. This function makes ls use columns if in a terminal and otherwise print one-per-line, while tricking ls into printing spaces verbatim because it's running through a pipe. ls() {     # only way I can stop ls from escaping with backslashes     if [ -t 1 ]; then         /system/bin/ls -C $@ |cat     else         /system/bin/ls $@ |cat     fi }
101990
I have a file that contains a list of names. i.e.:  Long Name One (001) Long Name Two (201) Long Name Three (123) ...   with spaces and some special characters. I wanted to make directories out of these names, i.e.:  cat file | xargs -l1 mkdir   It makes individual directories separated by spaces, i.e. Long, Name, One, Two, Three, instead of  Long Name One (001), Long Name Two (201), Long Name Three (123).  How can I do that?
34780
I want to know which files have the string $Id$.  grep \$Id\$  my_dir/mylist_of_files   returns 0 occurrences.  I discovered that I have to use  grep \$Id$ my_dir/mylist_of_files  Then I see that the $Id is colored in the output, i.e. it has been matched.  How could I match the second $ and why doesn't \$Id\$ work.  It doesn't matter if the second $ is the last character or not.  I use grep 2.9.    Before posting my question, I used google...       To search for a $ (dollar sign) in the file named test2, enter:      grep \\$ test2      The \\ (double backslash) characters are necessary in order   to force the shell to pass a \$ (single backslash, dollar sign) to the   grep command. The \ (single backslash) character tells the grep   command to treat the following character (in this example the $) as a   literal character rather than an expression character. Use the fgrep   command to avoid the necessity of using escape characters such as the   backslash.   but I don't understand why grep \$Id works and why grep \\$Id\\$ doesn't.  I'm a little bit confused...
39631
I am trying to download files from .  The URL is:   When I use this command:  wget http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&amp;format=file    I get only index.html?acc=GSE48191 which is some kind of binary format.  How can I download the files from this HTTP site?
21059
So I'm relatively new to command line. I was able to use find to get an output of multiple files from multiple directories since there was no specific place these would be (I'm sure this can be shortened):  find ./ -name filename1.ext &amp;&amp; find ./ -name filename2.ext &amp;&amp; find ./ -name filename3.ext   Now that gave me the list of what I was wanting but now that I've found the files in question, I want to grep them for information.
19129
I have been using Wget, and I have run across an issue. I have a site,that has several folders and subfolders within the site. I need to download all of the contents within each folder and subfolder. I have tried several methods using Wget, and when i check the completion, all I can see in the folders are an "index" file.  I can click on the index file, and it will take me to the files, but i need the actual files.  does anyone have a command for Wget that i have overlooked, or is there another program i could use to get all of this information?  site example:  www.mysite.com/Pictures/ within the Pictures DIr, there are several folders.....  www.mysite.com/Pictures/Accounting/  www.mysite.com/Pictures/Managers/North America/California/JoeUser.jpg  I need all files, folders, etc.....
55496
When using find, how do I return the file name and the line number when searching for a string? I manage to return the file name in one command and the line numbers with another one, but I can't seem to combine them.  File names: find . -type f -exec grep -l 'string to search' {} \;  Line numbers: find . -type f -exec grep -n 'string to search' {} \;
111951
The idea would be to use it as... a pipe in a command. For instance: say there's some kind of long path which has to be retyped again and again, followed by a pipe and a second program, i.e.  "directory1/directory2/direcotry3/file.dat | less -I "   I'd like that part to be stored in a variable, so it could be used like this:  r="directory1/directory2/direcotry3 \| less -I -p " $ cat path1/path2/$r &lt;searchterm&gt;   Instead, I get  cat: invalid option -- I Try `cat --help' for more information.   ... meaning the pipe clearly didn't work.
49924
I put together a script to do some file operations for me. I am using the wild card operator * to apply functions to all files of a type, but there is one thing I don't get. I can unzip all files in a folder like this  unzip "*".zip   However, to remove all zip files afterward, I need to do   rm *.zip   That is, it does not want the quotation marks. The unzip, on the other hand, does not work if I just give it the * (gives me a warning that "files were not matched").   Why is this different? To me, this seems like the exact same operation. Or am I using the wild card incorrectly?   in Unix do not really go into this, and I could not locate anything in the rm or zip docs.  I am using the terminal on a Mac (Yosemite).
44386
I have a directory filled with various files, differently named (i.e. not named with a specific pattern) and with different extensions, and I want to put each file in a newly created subdirectory named after each file; what's the best way to accomplish that from the command line under Linux? This sounds a bit confusing, let me make and example: let's call a random file in the directory ./filename.ext, I'd like to put it in a subdirectory called ./filename so that effectively the new path will be ./filename/filename.ext.  I did some trials with the following steps:   I've created some test files, from file-1.txt to file-9.txt, inside a test directory with for num in {1..9}; do touch file-$num.txt; done  I've created a directory named after each filename with for name in $(eval ls | cut -b 1-6); do mkdir $name; done I've moved the source files in their respective directories with for name in $(eval ls | grep -e ".txt" | cut -b 1-6); do mv $name.txt $name/$name.txt; done   As you can see, this solution works, but its huge drawback is that it works just in this case, with files of that extact filename lenght and with that particular extension. If I have some files with different names and extensions, this would be of no use at all. The use of cut command is not the ideal solution here, I suppose, but I haven't found anything better at the moment.  As a bonus, it would be cool to find a way to create the subdirs automatically when moving/renaming the files, without having to use mkdir previously (so when I ls to actually rename, I don't have to worry about excluding the newly created directories); but in the end this is a secondary problem.
50300
I want to list all files in a directory that start with 'abc' and end with '.zip'  I'm trying to use ls.  The directory has a lot of zip files starting with abc&lt;date&gt; and xvz&lt;date&gt;. I only want to get the list of the abc&lt;date&gt;.zip
11740
I'm running tests of a variable-size-and-contents test set. Data files are added and removed frequently. I'm looking for an automated way of gathering a file list.  All files are in subdirectories of D; I need the full directory and name added to a text file. However, I only need those files that have a "paired" file, which has the same filename but a different extension (so... different filename, but in a structured way). So, if there is a MyFileName.A and MyFileName.B, then I want D/.../MyFileName added to the file list.  There are .A files without .B files, but no .B files without .A files. If a .A has a .B file, then both files are in the same directory.   Any advice?
83149
If I have a file which name starting with single or several minus sign, for example --1 it can't be used as a parameter of many commands. Even if I run  cat --1   instead of file content I get unrecognised option error message:  cat: unrecognized option '--1'   Same effect appears when I type  cat "--1" cat '--1' cat \-\-1   nothing works. Files starting with '-' is not illegal in file system, but it is quite hard to work with them in cli and scripts. OK, in case of cat or grep I can use  cat &lt;--1   and this will work, but  rm --1   won't work either and it is quite hard to substitute this command with something else. Very uncomfortable after all. Is there any universal workaround different from not using such file names?   BTW, if name file is single - and all most of commands will understand it as stdin, wouldn't they?   And it would be hard not to use such file names since I can't rename them all in script using mv command either.
84640
Possible Duplicate:          I'm having some confusion with the find command.  If I type:  find . -name *.xml   or   find ./ -name *.xml   I receive a list of all the xml files in subdirectories of the current directory, UNLESS the current directory contains an xml file, in which case I get:  find: paths must precede expression: foo.xml Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]   What is the correct way to search for files with wildcard when the current directory may contain a match?
89391
I have to download a file from this . The file download is a zip file which I will have to unzip in the current folder.  Normally, I would download it first, then run the unzip command.  $ wget http://www.vim.org/scripts/download_script.php?src_id=11834 -O temp.zip $ unzip temp.zip   But in this way, I need to execute two commands, wait for the completion of first one to execute the next one, also, I must know the name of the file temp.zip to give it to unzip.  Is it possible to redirect output of wget to unzip? Something like  $ unzip &lt; `wget http://www.vim.org/scripts/download_script.php?src_id=11834`   But it didn't work.      bash: wget   http://www.vim.org/scripts/download_script.php?src_id=11834   -O temp.zip: ambiguous redirect   Also, wget got executed twice, and downloaded the file twice.
86035
If I do ls -1 target_dir | wc -l, I get a count of files in a directory. I find this a bit cumbersome. Is there a more elegant or succinct way?
28963
I thought this would be simple - but it is proving more complex than I expected.  I want to iterate through all the files of a particular type in a directory, so I write this:  #!/bin/bash  for fname in *.zip ; do    echo current file is ${fname} done   This works as long as there is at least one matching file in the directory. However if there are no matching files, I get this:  current file is *.zip   I then tried:  #!/bin/bash  FILES=`ls *.zip` for fname in "${FILES}" ; do     echo current file is ${fname} done   While the body of the loop does not execute when there are no files, I get an error from ls:  ls: *.zip: No such file or directory   How do I write a loop which cleanly handles no matching files?