I'm trying to read huge hive table with 13 million records using groovy where the data is in parquet format. I used following to code but I'm getting OOM java heap space error. I have given max 32 GB memory and setFetchsize(5000) still its getting error.  JAVA_OPTS="-Xms1024M" JAVA_OPTS="-Xmx32556M"   Any Help will be appreciated.  Code:    String contSql = "select * from staging.cont_staging";                 ResultSet resRateRecords = stmt.executeQuery(contSql);                 Map &lt;String,Map&lt;String,String&gt;&gt; masterRecords = new HashMap&lt;String,Map&lt;String,String&gt;&gt;();                 Map&lt;String,String&gt; existingRecords = null;                 int count = 0;                 resRateRecords.setFetchSize(5000);                 while(resRateRecords.next()) {                          try{existingRecords = new HashMap&lt;String,String&gt;();  masterRecords.put(resRateRecords.getString("contract_id")+"#"+count++,existingRecords);                         }catch(Exception e){                                 e.printStackTrace();                         }   Error  java.lang.OutOfMemoryError: Java heap space         at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:355)         at org.apache.thrift.protocol.TBinaryProtocol.readString(TBinaryProtocol.java:347)         at org.apache.hive.service.cli.thrift.TStringColumn$TStringColumnStandardScheme.read(TStringColumn.java:453)         at org.apache.hive.service.cli.thrift.TStringColumn$TStringColumnStandardScheme.read(TStringColumn.java:433)         at org.apache.hive.service.cli.thrift.TStringColumn.read(TStringColumn.java:367)         at org.apache.hive.service.cli.thrift.TColumn.standardSchemeReadValue(TColumn.java:328)         at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)         at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)         at org.apache.thrift.TUnion.read(TUnion.java:138)         at org.apache.hive.service.cli.thrift.TRowSet$TRowSetStandardScheme.read(TRowSet.java:573)         at org.apache.hive.service.cli.thrift.TRowSet$TRowSetStandardScheme.read(TRowSet.java:525)         at org.apache.hive.service.cli.thrift.TRowSet.read(TRowSet.java:451)         at org.apache.hive.service.cli.thrift.TFetchResultsResp$TFetchResultsRespStandardScheme.read(TFetchResultsResp.java:518)         at org.apache.hive.service.cli.thrift.TFetchResultsResp$TFetchResultsRespStandardScheme.read(TFetchResultsResp.java:486)         at org.apache.hive.service.cli.thrift.TFetchResultsResp.read(TFetchResultsResp.java:408)         at org.apache.hive.service.cli.thrift.TCLIService$FetchResults_result$FetchResults_resultStandardScheme.read(TCLIService.java:13251)         at org.apache.hive.service.cli.thrift.TCLIService$FetchResults_result$FetchResults_resultStandardScheme.read(TCLIService.java:13236)         at org.apache.hive.service.cli.thrift.TCLIService$FetchResults_result.read(TCLIService.java:13183)         at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)         at org.apache.hive.service.cli.thrift.TCLIService$Client.recv_FetchResults(TCLIService.java:505)         at org.apache.hive.service.cli.thrift.TCLIService$Client.FetchResults(TCLIService.java:492)         at org.apache.hive.jdbc.HiveQueryResultSet.next(HiveQueryResultSet.java:335)         at java_sql_ResultSet$next.call(Unknown Source)         at BEContractRateLoad.fetchContractRateRecords(DestRateLoad.groovy:300)         at BEContractRateLoad.processContractRecords(DestRateLoad.groovy:397)         at BEContractRateLoad$processContractRecords$1.call(Unknown Source) Groovy has reported an error, terminating
I am writing a client-side Swing application (graphical font designer) on Java 5. Recently, I am running into java.lang.OutOfMemoryError: Java heap space error because I am not being conservative on memory usage. The user can open unlimited number of files, and the program keeps the opened objects in the memory. After a quick research I found  and others saying on Windows machine the JVM defaults max heap size as 64MB.  Given this situation, how should I deal with this constraint?  I could increase the max heap size using command line option to java, but that would require figuring out available RAM and writing some launching program or script. Besides, increasing to some finite max does not ultimately get rid of the issue.  I could rewrite some of my code to persist objects to file system frequently (using database is the same thing) to free up the memory. It could work, but it's probably a lot work too.  If you could point me to details of above ideas or some alternatives like automatic virtual memory, extending heap size dynamically, that will be great.
Summer in Saint Helena The Winter Bash site is located at http://winterba.sh/.  Only one problem:   . Saint Helena is in the Southern Hemisphere. It's Summer in Saint Helena.   So shouldn't it be the Summer Bash? Don't forget our Aussie/Kiwi/South African/Southern Brazilian/Argentine/Uruguayan/Zimbabwean/Swazi/Tokelauan/Tuvaluan/Pitcairn Islander/ friends! How to cleanly launch a GUI app via the Terminal? Some GUI apps launch cleanly via the Terminal command line, but some don't, and they cause the Terminal to wait for the app to terminate. Even then, some don't "release" the command line.  The mysterious ampersand &amp; suffix seems to cause the terminal to put the process into the background (but I'm not sure what happens there).  Is there a way to launch an app via the Terminal so that there is no "hang on" effect, just like launching something via Alt+F2?  I'd like to have the command line available again immediately, without something still in the background and printing in the terminal. How to read an already existing NgSpice raw file This has taken me a couple of days to solve. I have a set of NgSpice raw files that I would like to exploit so as to visualize the voltage distributions in different circuits. What is the python command to use or python package to import in order for me to achieve this. Note: I am a bit familiar with the PySpice python package, but from what I have read and understood it works when you use it to create your netlist and run it, by using the already stored variables in its memory. In my case however, raw files are already generated... Hope I was clear enough. What to do with an Ubuntu disk image that appears as a WinRAR file? I've never used Ubuntu before. Just heard from some friends. "IT'S GREAT!!!", they used to say. So i decided to try it. I downloaded WUBI, and ran it but 'twas takin' too much time. So I decided to download the .ISO Image file separately. It couldn't download more after 150 mB, and got interrupted (and cancelled) itself. I just went to , and further went to , and searched for my question. After having found a related one, which told     "try the torrent way"   So I downloaded it through the torrent. And after that when I opened the .rar file, there wasn't any .ISO image file; just some folders with names like .disc, boot, ISOLINUX, etc. I searched for the .ISO file but nothin' there: ANYONE HELP ME, PLEASE!!! or this will drive me into never thinkin' about Ubuntu again. Apt-get update with an @ in password error Zero search results when searching for [tagOne] [or] [tagTwo] score:-2 I'm able to search for the newest questions tagged with [xcode] and [ios] and score >= -2.    But I see 0 results when searching for the newest questions tagged with [xcode] or [ios] and score >= -2. Why is the network configuration I set in /etc/network/interfaces ignored on Ubuntu 17.10? When a ball moves to the right, friction acts to oppose the motion, in other words, to the left. However, when a car travels around a bend, the friction acts in the perpendicular direction to the car's velocity and provides the centripetal force. I just cannot understand why friction would act in that direction. How was Hedwig able to find Sirius? In the beginning of Goblet of Fire, Harry sends Snuffles a letter.  Hedwig somehow finds him after the extremely specific command from Harry, "Just find him, okay?"   Theories:   Hedwig has a scanning system, like Baymax in Big Hero 6. Hedwig got super lucky. Hedwig flies really, really fast.   Now I know that owls are intelligent beings in the Harry Potter canon. But.... those ideas are just as crazy as the next.  So how did Hedwig do it? Plotting heat equation in a Manipulate expression How can I hide my code in visualforce page? I developed my app,   The controller is complete, and also the visualforce, but I need to hide my code at installing the app in another salesforce account.  How can I make this possible?  Is there any way to hide both codes? The apex and the code behind?   Thanks in advance. What kind of distribution is this "almost" uniformly distributed data for calls/week? My supervisor asked me to find out which distribution represents a particular situation.  I have a VoIP generator that generates calls "uniformly" distributed between callers. This means that the volume per caller distribution is "almost" uniformly distributed between a minimum and maximum. So by running a test with 10000 users and a min value equal to 30 calls per week and a max value equal to 90 calls/week i obtain that not all the users respect this limits: we have some users that generate &lt;30 calls and some other users that generate >90 calls. It is clear that the obtained distribution is not uniform.  The situation is this:    He said that i have to perform a sort of numerical process in order to find some formulas that could define this distribution. Initially, as wrote before,we wanted to obtain a uniform (min,max) distribution (the green area in the figure) but this is not the case as proved with chi-square test.  Moreover the curve in figure is not symmetric, the probability of call generation below 30 call/week and greater that 90 call/week are not identical (it is high for 90calls/week). The variability of the number of generated call increases with the increasing call generation rates.   "Actually implementation of this distribution is nothing but assigning different call rates in a range for users in domain which indicates implementation of several delta functions. As the call rates increases the variability of the generated calls also increases with the average call rate and this leads to the asymmetric behavior of the curve." [cit. from the Voipgenerator documentation]  Someone can help me?I think that now i cannot use Q-Q plot because i don't know which theoretical distribution i have to use in order to compare it with my empirical data.  Sorry if I have stressed with a similar problem a few weeks ago, but initially we thought we could change the implementation, but now we cannot. Hence i have to discover the type of the distribution i obtained and i don't know how can i do this. I'm currently working on a web application which has a page which displays a single chart (a .png image).  On another part of this page there are a set of links which, when clicked, the entire page reloads and looks exactly the same as before except for the chart in the middle of the page.  What I want to do is when a link is clicked on a page just the chart on the page is changed.  This will speed things up tremendously as the page is roughly 100kb large, and don't really want to reload the entire page just to display this.    I've been doing this via JavaScript, which works so far, using the following code  document.getElementById('chart').src = '/charts/10.png';   The problem is that when the user clicks on the link, it may take a couple of seconds before the chart changes.  This makes the user think that their click hasn't done anything, or that the system is slow to respond.  What I want to happen is display a spinner / throbber / status indicator, in place of where the image is while it is loading, so when the user clicks the link they know at least the system has taken their input and is doing something about it.    I've tried a few suggestions, even using a psudo time out to show a spinner, and then flick back to the image.    A good suggestion I've had is to use the following  &lt;img src="/charts/10.png" lowsrc="/spinner.gif"/&gt;   Which would be ideal, except the spinner is significantly smaller than the chart which is being displayed.  Any other ideas? $x^2-y^2=2s$, s cannot be an odd integer How can we prove that if $x^2-y^2=2s$ holds, s cannot be an odd integer. What theorem in number theory should we use? Can I travel to Paris for a day with a single entry visa for Schengen from Switzerland? I am visiting a friend in Switzerland for ten days. I have already got visitor's visa for the same on my Indian passport. Particulars mentioned on my visa are:  1. 'Etats Schengen' , 2. Single entry 3. Dates of visit. &amp; 4. Type of visa : C.  I understand that having already got ‘Etats Schengen' visa, I can travel to all Schengen countries.  I am also planning to visit Paris for a day during my stay in Switzerland. However, I never mentioned this in my visa application. I am planning to take a train from Geneva to Paris and book a hotel in Paris for one night. Can I possibly plan my Paris trip this way as I have Schengen visa, or I am not allowed to travel out of Switzerland as I never mentioned this in my visa application?   Also what does Single entry on my Schengen visa means: does it mean that I can enter and leave Switzerland only once or it means that I can enter and leave the Schengen region only once. Formula for prime counting function I saw this formula on  page 2  $$\pi (n)=\sum_{j=2}^{n}\frac{\sin^{2}\left(\pi \frac{(j-1)!^{2}}{j}\right)}{\sin^{2}(\frac{\pi }{j})}$$  Where $\pi(n)$ is the prime counting function. Is this true? How to prove it? How to prove that $\lim\limits_{h \to 0} \frac{a^h - 1}{h} = \ln a$ In order to find the derivative of a exponential function, on its general form $a^x$ by the definition, I used limits.  $\begin{align*} \frac{d}{dx} a^x &amp; = \lim_{h \to 0} \left [ \frac{a^{x+h}-a^x}{h} \right ]\\  \\ &amp; =\lim_{h \to 0} \left [ \frac{a^x \cdot a^h-a^x}{h} \right ] \\  \\  &amp;=\lim_{h \to 0} \left [ \frac{a^x \cdot (a^h-1)}{h} \right ] \\  \\ &amp;=a^x \cdot \lim_{h \to 0} \left [\frac {a^h-1}{h} \right ] \end{align*}$  I know that this last limit is equal to $\ln(a)$ but how can I prove it by using basic Algebra and Exponential and Logarithms properties? Thanks Retrieve malicious IP addresses from Apache logs and block them with iptables What is the CVE-2014-6271 bash vulnerability (Shellshock) and how do I fix it? Recently, there have been news going around regarding "CVE-2014-6271" (See ), which is a vulnerability in Bash. How do I know if I am affected by this, how can I fix it, and why should I care?  This is designed as a canonical answer for this vulnerability, due to its scope and severity. VirtualBox Toolbar is hidden! How to show it? I accidentally hide the main Toolbar from my VirtualBox Manager UI? In my ubuntu 12.04 .   Can anyone let me know the detail steps to display the Toolbar again? I search the whole blogs but can't figure it out!