I would like to create a java program which would poll a log file infinitely. Whenever there is some entry into that logfile, i would like to read the latest contents into the file just written.   can anyone please let me know the efficient method to do this? as this should run 24/7 getting cpu spikes is not desirable.   i did my basic research and found the following but didnt help much:
15568
I'm wondering what techniques and/or library to use to implement the functionality of the linux command "tail -f ".  I'm essentially looking for a drop in add-on/replacement for java.io.FileReader.  Client code could look something like this:  TailFileReader lft = new TailFileReader("application.log"); BufferedReader br = new BufferedReader(lft); String line; try {   while (true) {     line= br.readLine();     // do something interesting with line   } } catch (IOException e) {   // barf }   The missing piece is a reasonable implementation of TailFileReader.  It should be able to read parts of the file that exist before the file is opened as well as the lines that are added.
6610
When copying large files using shutil.copy(), you get no indication of how the operation is progressing..  I have put together something that works - it uses a simple ProgressBar class (which simple returns a simple ASCII progress bar, as a string), and a loop of open().read() and .write() to do the actual copying. It displays the progress bar using sys.stdout.write("\r%s\r" % (the_progress_bar)) which is a little hackish, but it works.  You can see the code (in context)   Is there any built-in module that will do this better? Is there any improvements that can be made to this code?
64577
I want the log to roll over as long as the application is running, but I want the log to start fresh when the application is restarted.  Updated: Based on  feedback, my appender looks like this:     &lt;appender name="myRFA" class="org.apache.log4j.RollingFileAppender"&gt;       &lt;param name="File" value="my-server.log"/&gt;       &lt;param name="Append" value="false" /&gt;       &lt;param name="MaxFileSize" value="10MB"/&gt;       &lt;param name="MaxBackupIndex" value="10"/&gt;       &lt;layout class="org.apache.log4j.PatternLayout"&gt;          &lt;param name="ConversionPattern"             value="%d{ISO8601} %p - %t - %c - %m%n"/&gt;       &lt;/layout&gt;    &lt;/appender&gt;   I simply added the following line:  &lt;param name="Append" value="false" /&gt;   It now truncates the base log file at startup, but it leaves the rolled files alone.
25296
Does anybody have a snippet of Java that can return the newest file in a directory (or knowledge of a library that simplifies this sort of thing)?
8442
I have a file (called "number.txt") which I want to read to an array in Java. How exactly do I go ahead and do this? It is a straight-forward "1-dimensional" file, containing 100 numbers.  The problem is that I get an exception every time. Apparently it can't find it (I am sure its spelled correctly). When looking through code examples, it doesn't specify the file's entire file path, only the name of the file itself. How would I go about doing that if its necessary?  Also, when reading the file, will the array automatically contain all the lines of the file, or will I have to make a loop which which copies every line to corresponding subscript i?  I've heard of BufferedReader class, what it's purpose, and how does it corelate to reading input?
109072
This is more a conceptual thing than a current problem.  Would it be possible to do this:  tail -f &lt;file&gt; | grep "string" | sed 's/stuff//g' &gt;&gt; output.txt  The problem I'm thinking is that tail -f never terminates so it won't progress in the command. I'm trying to read and output a file in real-time. If this doesn't work how would I do it?
71595
It has always bothered me that the only way to copy a file in Java involves opening streams, declaring a buffer, reading in one file, looping through it, and writing it out to the other steam.  The web is littered with similar, yet still slightly different implementations of this type of solution.  Is there a better way that stays within the bounds of the Java language (meaning does not involve exec-ing OS specific commands)?  Perhaps in some reliable open source utility package, that would at least obscure this underlying implementation and provide a one line solution?
37002
I'm trying to watch for any new output of a log file. Another script (not under my control) is deleting the file then creating a new one with the same name. Using tail -f doesn't work because the file is being deleted.
104676
I have a Java application that I run from a console which in turn executes an another Java process. I want to get a thread/heap dump of that child process.  On Unix, I could do a kill -3 &lt;pid&gt; but on Windows AFAIK the only way to get a thread dump is Ctrl-Break in the console. But that only gives me the dump of the parent process, not the child.  Is there another way to get that heap dump?
314
I have no idea where to start looking. I've been reading about daemons and didn't understand the concept.  More details :   I've been writing a crawler which never stops and crawlers over RSS in the internet. The crawler has been written in java - therefore its a jar right now. I'm an administrator on a machine that has Ubuntu 11.04 . There is some chances for the machine to crash , so I'd like the crawler to run every time you startup the machine. Furthermore, I'd like it to keep running even when i logged out. I'm not sure this is possible, but most of the time I'm logged out, and I still want to it crawl.   Any ideas? Can someone point me in the right direction?  Just looking for the simplest solution.
44279
I am trying to print every Nth line out of a file with more than 300,000 records into a  new file. This has to happen every Nth record until it reaches the end of the file.
11052
I've got a long-running command that generates a lot of output on stdout. I'd like to be able to retain, for instance, only the last three days or the last gibibyte (avoiding cutting lines in the middle), and, if possible, in file chunks not larger than 20&nbsp;MiB. Each file chunk is being named with a numeric suffix or a timestamp.  Something like:  my-cmd | magic-command --output-file-template=my-cmd-%t \                        --keep-bytes=1G \                        --keep-time=3d \                        --max-chunk-size=20M \                        --compress=xz   Would write:  my-cmd-2014-09-05T10:04:23Z   When it's reaching 20M, it would compress it and open a new one, and so on, and after a while would it starts deleting the oldest files.  Does such a command exist?  I'm aware of logrotate and its ability to manage files written by other applications, but I'm looking for something simpler that doesn't involve having to set up a cron job, specify rules, suspend the process, etc.
36624
I have a growing log file for which I want to display only the last 15 lines. Here is what I know I can do:  tail -n 15 -F mylogfile.txt   As the log file is filled, tail appends the last lines to the display.  I am looking for a solution that only displays the last 15 lines and get rid of the lines before the last 15 after it has been updated. Would you have an idea?
6379
I want to create a log file for a cron script that has the current hour in the log file name. This is the command I tried to use:  0 * * * * echo hello &gt;&gt; ~/cron-logs/hourly/test`date "+%d"`.log   Unfortunately I get this message when that runs:  /bin/sh: -c: line 0: unexpected EOF while looking for matching ``' /bin/sh: -c: line 1: syntax error: unexpected end of file   I have tried escaping the date part in various ways, but without much luck. Is it possible to make this happen in-line in a crontab file or do I need to create a shell script to do this?
101758
Is there a way to write log4j logging events to a log file that is also being written to by other applications. The other applications could be non-java applications. What are the drawbacks? Locking issues? Formatting?
46341
I'm writing a log file viewer for a web application and for that I want to paginate through the lines of the log file.  The items in the file are line based with the newest item at the bottom. So I need a tail() method that can read n lines from the bottom and support an offset.  This is hat I came up with: def tail(f, n, offset=0):     &quot;&quot;&quot;Reads a n lines from f with an offset of offset lines.&quot;&quot;&quot;     avg_line_length = 74     to_read = n + offset     while 1:         try:             f.seek(-(avg_line_length * to_read), 2)         except IOError:             # woops.  apparently file is smaller than what we want             # to step back, go to the beginning instead             f.seek(0)         pos = f.tell()         lines = f.read().splitlines()         if len(lines) &gt;= to_read or pos == 0:             return lines[-to_read:offset and -offset or None]         avg_line_length *= 1.3  Is this a reasonable approach?  What is the recommended way to tail log files with offsets?
14169
I want to monitor a file. So I have tailed a file using below command, to execute a script per line.  tail -3f logfile.log | grep "action.*add" | sed -u -e "s/^/'/" -e "s/$/'/" | xargs -L 1 -P 5 bash testscript.sh   But it seems that script is not getting executed.  I observed that grep is not giving any input for next pipe.  When I tried,  tail -3f logfile.log | grep "action.*add"   it worked. But when given next filter like sed, grep, xargs etc. It didn't worke like the one shown below.  tail -3f /var/tmp/rabbitmq-tracing/logfile.log | grep "action.*add" | grep add    Please help me to know why this is happening and how to overcome this.    Edit 1: Basically anything like below should work and it was working previously. Confused why it is not working now.  tail -f file.txt | grep something | grep something | grep something     EDIT 2: Output line after first grep will be a json string like below. And I want to give this line as input( enclosed in single quotes) to bash script.  {"twNotif": {"originator": "api", "chain": "test", "txId": "08640-0050568a5514", "version": "1.0", "msgType": "api", "twData": {"api": {"hostId": "007bdcc5", "user": "test", "cmdTxt": "100599"}}, "action": "add", "store": "test", "msgTime": 1467280648.971042}}