I’m reading large files containing DNA sequences. These are long stretches of characters and I need a certain subset from somewhere in the file (I have the start- and stop position). Since the files are so big, I use BufferedReader() to read. This reads one line at a time but the subset I want might be on more than one.  The start- and stop positions I have only make sense if the whole DNA sequence was represented as one line with no newlines. In practise, these files do contain newlines unfortunately. So for every line the indices are from 0 to end and not 0 to 20, 21 to 40, 41 to 60 etc. for example.  My problem/question: Reading a file line by line, but saving a subset/substring that might be across multiple lines. I tried several methods but can’t extract the substring I want. I suspect my own logic/thinking is flawed, or there is a method I am not yet aware of. Is there a better way to do this?  Method 1:  public String getSubSequence() {           fileLocation = "genome.fna";         String referenceGenomeSub = "";         int passedLetters = 0;         int passedLines = 0;          //start- and stop position         int start = 50;         int stop = 245;          Path path = Paths.get(fileLocation);            try (BufferedReader br = Files.newBufferedReader(path, Charset.defaultCharset())){              String line;              while ((line = br.readLine()) != null) {                  if (!line.startsWith("&gt;")) {//Don't need lines that start with &gt;                      passedLines++;                      //edit indices so I don't get out of bounds                     if (linesPassed != 1) {                         start = start - passedLetters;                         stop = stop - passedLetters;                     }                       //this is to know where I am in the file                     passedLetters = passedLetters + line.length();                       //if the subset is on only one line                     if (start &lt; passedLetters &amp;&amp; stop &lt;= passedLetters) {                                                 referenceGenomeSub = referenceGenomeSub.concat(line);                                             }                       //subsequence is on multiple lines                     else if (start &lt;= passedLetters &amp;&amp; stop &gt; passedLetters) {                         referenceGenomeSub = line.substring(start);                     }                     else if (passedLetters &gt; stop &amp;&amp; !referenceGenomeSub.isEmpty()) {                         referenceGenomeSub = referenceGenomeSub.concat(line.substring(0, stop));                     }                  }              }             br.close();          } catch (IOException e) {             System.out.println("Error: " + e.getMessage());         }      } }   Here I try to keep track of the number of characters I already have passed. This how I know when I’m in range of the desired substring. Result: StringIndexOutOfBoundsException  My other method is to save all lines up until the line with my stop position. I then extract a substring. This is not prefered as my esired subset might be at the end of the file.  Conditions:  - Memory friendly  - No BioJava if possible. I’m still learning to program so I want to do this without. Even if it’s the hard way  Not looking for fixed code but some article/example to get me on the right track is perfectly fine. I'm looking at my screen for hours now without making progress and my mind is a bit of a blank now. As I said, the problem might be flawed thinking or being oblivious to much better methods/techniques for this job.
93
I've been using the == operator in my program to compare all my strings so far. However, I ran into a bug, changed one of them into .equals() instead, and it fixed the bug.  Is == bad? When should it and should it not be used? What's the difference?
14412
I have created a function wherein I split a string from the csv filecsvAsString.split('\n') . It is working when the string is not so big. But when I try to split a very large string, an error will occur      regex too complicated   what I want to do is to catch if an error occur during the split process. How will I do it ? I've tried to use try and catch but the problem still occur. Is there any other possible solution?
50127
What would be the most efficient method of reading a text file into a dynamic one-dimensional array? reallocing after every read char seems silly, reallocing after every read line doesn't seem much better. I would like to read the entire file into the array. How would you do it?
19725
Consider the following example. If I do \seqsplit{ReferencedIDs} the string does not get split. If on the other hand, I insert the string as an argument directly it does. As usual, I don't know why this isn't working. Can anyone enlighten me?  \documentclass[12pt]{article} \usepackage{seqsplit} \begin{document} \gdef \ReferencedIDs{,2014.11.14,2014.11.14,2014.11.17,2014.11.14,2014.07.18.tm,2014.11.12.tm,2013.10.05.kanjur,2013.10.05.powai,2013.10.06,2013.10.28.zhm,2013.10.17,2013.10.22,2013.10.14.ks,2013.10.06,2013.10.17,2013.10.22,2013.10.24,2013.10.28,2013.10.30,2013.11.11,2014.09.05.tm,2014.07.18.tm,2014.11.12.tm,2014.11.17,2015.01.06,2015.01.06}  \seqsplit{\ReferencedIDs}   \seqsplit{,2014.11.14,2014.11.14,2014.11.17,2014.11.14,2014.07.18.tm,2014.11.12.tm,2013.10.05.kanjur,2013.10.05.powai,2013.10.06,2013.10.28.zhm,2013.10.17,2013.10.22,2013.10.14.ks,2013.10.06,2013.10.17,2013.10.22,2013.10.24,2013.10.28,2013.10.30,2013.11.11,2014.09.05.tm,2014.07.18.tm,2014.11.12.tm,2014.11.17,2015.01.06,2015.01.06}  \end{document}
71270
How would you perform a grep for text that appears on two lines?   For example:  pbsnodes is a command I use that returns the utilization of a linux cluster  root$ pbsnodes node1     state = free     procs = 2     bar = foobar  node2     state = free     procs = 4     bar = foobar  node3     state = busy     procs = 8     bar = foobar   I want to determine the number of procs that match nodes that are in state 'free'. So far I have been able to determine the "number of procs" and "the nodes in free state", but I want to combine them into one command that shows all free procs.   In the above example, the correct answer would be 6 (2+4).   What I have  root$ NUMBEROFNODES=`pbsnodes|grep 'state = free'|wc -l` root$ echo $NUMBEROFNODES 2  root$ NUMBEROFPROCS=`pbsnodes |grep "procs = "|awk  '{ print $3 }' | awk '{ sum+=$1 } END { print sum }'` root$ echo $NUMBEROFPROCS 14   How can I search for every line that reads 'procs = x', but only if the line above it reads 'state = free?
6617
I want to loop over the contents of a text file and do a search and replace on some lines and write the result back to the file. I could first load the whole file in memory and then write it back, but that probably is not the best way to do it.  What is the best way to do this, within the following code?  f = open(file) for line in f:     if line.contains('foo'):         newline = line.replace('foo', 'bar')         # how to write this newline back to the file
15568
I'm wondering what techniques and/or library to use to implement the functionality of the linux command "tail -f ".  I'm essentially looking for a drop in add-on/replacement for java.io.FileReader.  Client code could look something like this:  TailFileReader lft = new TailFileReader("application.log"); BufferedReader br = new BufferedReader(lft); String line; try {   while (true) {     line= br.readLine();     // do something interesting with line   } } catch (IOException e) {   // barf }   The missing piece is a reasonable implementation of TailFileReader.  It should be able to read parts of the file that exist before the file is opened as well as the lines that are added.
11442
I have an application that writes information to file. This information is used post-execution to determine pass/failure/correctness of the application. I'd like to be able to read the file as it is being written so that I can do these pass/failure/correctness checks in real time.  I assume it is possible to do this, but what are the gotcha's involved when using Java? If the reading catches up to the writing, will it just wait for more writes up until the file is closed, or will the read throw an exception at this point? If the latter, what do I do then?   My intuition is currently pushing me towards BufferedStreams. Is this the way to go?
76807
Intro  I'm working on a lightweight package that allows you to put any content (typically LaTeX code) in an environment, then use that content multiple times, each time scanned separately, so you can use it both as verbatim output and interpreted as LaTeX.  Its main purpose is to typeset LaTeX code and its 'result' side-by-side; useful for documenting TeX packages without having to copy/paste code. But it has other purposes as well.  After experimenting with \scantokens until I gave up in tears, I arrived at the filecontents environment, which almost solved my problem singlehandedly!  The Problem  The problem is, when reintroducing filecontented code through \input, LaTeX adds an implicit newline at the end, which translates to whitespace that I can't seem to get rid of.  I've seen many semi-related questions floating around, with answers employing techniques like:   reading a file line-by-line, using \everyeof in some clever way, playing with \endlinechar or using packages such as catchfile   but none of them provides a clear answer to the following question:  The Question  How can I \input a file without getting the implicit newline?  Minimal Working Example  \documentclass[margin=1mm]{standalone}  \usepackage{filecontents}  \begin{filecontents*}{tmp.tex}     \LaTeX{}\end{filecontents*}  \begin{document}     \fbox{\input{tmp.tex}} \end{document}     Note the extra whitespace to the right of the box.  Note also that I'm definitely not adding any explicit whitespace. Even a manually produced file (so, not using filecontents) with no newlines at all exhibits the same problem.
7366
How do I read a file into a std::string, i.e., read the whole file at once?  Text or binary mode should be specified by the caller. The solution should be standard-compliant, portable and efficient. It should not needlessly copy the string's data, and it should avoid reallocations of memory while reading the string.  One way to do this would be to stat the filesize, resize the std::string and fread() into the std::string's const_cast&lt;char*&gt;()'ed data(). This requires the std::string's data to be contiguous which is not required by the standard, but it appears to be the case for all known implementations. What is worse, if the file is read in text mode, the std::string's size may not equal the file's size.  A fully correct, standard-compliant and portable solutions could be constructed using std::ifstream's rdbuf() into a std::ostringstream and from there into a std::string. However, this could copy the string data and/or needlessly reallocate memory.    Are all relevant standard library implementations smart enough to avoid all unnecessary overhead?  Is there another way to do it?  Did I miss some hidden Boost function that already provides the desired functionality?     void slurp(std::string&amp; data, bool is_binary)
38733
I want to know if there is an easy way to split a line in a file into multiple lines, such as this:  I have  A B C 1 2 3 4   And I want to get something like this, based on the first string of the line:  A B A C 1 2 1 3 1 4   Basically based on the first string in the line create multiple lines with the second and third and fourth string and so on.
46341
I'm writing a log file viewer for a web application and for that I want to paginate through the lines of the log file.  The items in the file are line based with the newest item at the bottom. So I need a tail() method that can read n lines from the bottom and support an offset.  This is hat I came up with: def tail(f, n, offset=0):     &quot;&quot;&quot;Reads a n lines from f with an offset of offset lines.&quot;&quot;&quot;     avg_line_length = 74     to_read = n + offset     while 1:         try:             f.seek(-(avg_line_length * to_read), 2)         except IOError:             # woops.  apparently file is smaller than what we want             # to step back, go to the beginning instead             f.seek(0)         pos = f.tell()         lines = f.read().splitlines()         if len(lines) &gt;= to_read or pos == 0:             return lines[-to_read:offset and -offset or None]         avg_line_length *= 1.3  Is this a reasonable approach?  What is the recommended way to tail log files with offsets?
54110
I have a very big file 4GB and when I try to read it my computer hangs. So I want to read it piece by piece and after processing each piece store the processed piece into another file and read next piece.  Is there any method to yield these pieces ?  I would love to have a lazy method.
70238
In the upcoming Java7, there is a  to check if two file object are same file reference.  Are there similar API provided in the .NET framework?  I've search it over MSDN but nothing enlighten me.  I want it simple but I don't want to compare by filename which will cause problems with hard/symbolic links and different style of path. (e.g. \\?\C:\, C:\).   What I going to do is just prevent duplicated file being drag and dropped to my linklist.
33653
I have a text in a text file, where I want everything that is between strings like \{{[} and {]}\} to be deleted - including these strings themselves.  These two string can lie on different lines as well as on the same line. In either case,  on the line on which the beginning \{{[} lies, I don't want the text before, i.e. left, of it to be deleted - and the same holds for the text after {]}\}.  Here is an example: Given a text file with the content   Bla Bla bla bla \{{[} more bla bla even more bla bla bla bla.   A lot of stuff might be here.  Bla bla {]}\} finally done.  Nonetheless, the \{{[} show {]}\} goes on.   the script should return another text file with the content  Bla Bla bla bla  finally done.  Nonetheless, the  goes on.   Unfortunately this simple-looking task turned out to be too difficult for me to do with sed. I'm happy with any solution in any language, as long as I don't have to install anything on my standard linux machine (C and some java is already installed).
74252
I am writing a C library that reads a file into memory. It skips the first 54 bytes of the file (header) and then reads the remainder as data. I use fseek to determine the length of the file, and then use fread to read in the file.  The loop runs once and then ends because the EOF is reached (no errors). At the end, bytesRead = 10624, ftell(stream) = 28726, and the buffer contains 28726 values. I expect fread to read 30,000 bytes and the file position to be 30054 when EOF is reached.  C is not my native language so I suspect I've got a dumb beginner mistake somewhere.  Code is as follows:  const size_t headerLen = 54;  FILE * stream; errno_t ferrno = fopen_s( &amp;stream, filename.c_str(), "r" ); if(ferrno!=0) {   return -1; }  fseek( stream, 0L, SEEK_END ); size_t bytesTotal = (size_t)(ftell( stream )) - headerLen; //number of data bytes to read size_t bytesRead = 0; BYTE* localBuffer = new BYTE[bytesTotal]; fseek(stream,headerLen,SEEK_SET); while(!feof(stream) &amp;&amp; !ferror(stream)) {     size_t result = fread(localBuffer+bytesRead,sizeof(BYTE),bytesTotal-bytesRead,stream);     bytesRead+=result; }     Depending on the reference you use, it's quite apparent that adding a "b" to the mode flag is the answer. Seeking nominations for the bonehead-badge. :-)   talks about it in the second paragraph, second sentence (though not in their table).   doesn't discuss the binary flag until halfway down the page.   mentions the existance of the "b" tag, but states that it "shall have no effect".
88829
I want to write a program for this: In a folder I have n number of files; first read one file and perform some operation then store result in a separate file. Then read 2nd file, perform operation again and save result in new 2nd file. Do the same procedure for n number of files. The program reads all files one by one and stores results of each file separately. Please give examples how I can do it.