I'm using an ETL software Pentaho (only mildly relevant) -- it's built on Java. I'm getting heap space errors after 80 loops of a process every time --- I can increase the memory allocated to Java -xmx2048 or what have you, but that just "delays" the error and is a band-aid fix.  I have VisualVM and MemoryAnalyzer but am trying to figure out where to find the root causes/ objects/ threads that are causing extraneous objects to persist. I can see in VisualVM that the heap space memory gradually increases over time. I'm not sure what exactly I should be doing to find the cause though. Analyzing the heap dump upon error?
14268
I am writing a client-side Swing application (graphical font designer) on Java 5. Recently, I am running into java.lang.OutOfMemoryError: Java heap space error because I am not being conservative on memory usage. The user can open unlimited number of files, and the program keeps the opened objects in the memory. After a quick research I found  and others saying on Windows machine the JVM defaults max heap size as 64MB.  Given this situation, how should I deal with this constraint?  I could increase the max heap size using command line option to java, but that would require figuring out available RAM and writing some launching program or script. Besides, increasing to some finite max does not ultimately get rid of the issue.  I could rewrite some of my code to persist objects to file system frequently (using database is the same thing) to free up the memory. It could work, but it's probably a lot work too.  If you could point me to details of above ideas or some alternatives like automatic virtual memory, extending heap size dynamically, that will be great.
94736
free -m shows me the following in my system:               total       used       free     shared    buffers     cached Mem:          3954       3842        111          0        248       2585 -/+ buffers/cache:       1008       2945 Swap:         3811          4       3807   So memory looks pretty used up, but top sorted by memory shows the following top-processes:    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                8891 minecraf  20   0 2374m 277m 9.9m S    1  7.0   0:12.21 java                                                                                                                                                                    8704 root      39  19 2802m 272m  11m S    0  6.9   0:28.73 java         The virtual memory used by java seems to be huge, but i learned on the internet that that's not a problem. So i guess my question is: Why does free show the virtual memory, if it's not that meaningful?
70083
I'm trying to find out how much memory my objects take to see how many of them are ending up on the  (which is anything over 85,000 bytes). Is it as simple as adding 4 for an int, 8 for a long, 4 (or 8 if you're on 64 bit) for any reference types etc for each object, or are there overheads for methods, properties, etc.?
20332
i just made a test, to see how much data i can put in a string before i reach the the heapsize limit and i get an unexpected behaviour   string  csvFile = '"XXXXXXX","XXPX XXRXXXX","XXPX BP","XLXXXXR","XXXXVXXX PRXXXDXXXX","XXPX XXXXVXXX","XDRXXXDXX LXGXLX","XXXXRX XXLX","LXXXX XX","DXXX XXXZXX XXXXVXXX","DXXX XXXX XXXXVXXX","XXDXXX BP","XXDXXX XXXXXLX","PXRXXXX XVX","DXXXXXXXZXXXX","XXXX XXRXXXXRX","XXGXXXX XXRXXXXRX","DXXX DX XXXXXXX","LXXGX DX XXXXXXX","PXD","XXXXX XXXXRXXXXXLX","XXXX RXXXPXXX","XXDXRXZZX RXXXPXXX","PRXVXXXXX RXXXPXXX","XXXXXX RXXXPXXX","XXP RXXXPXXX","XXXXRX XXLXXXXX 1","XXXXRX XXLXXXXX 2","XXXXRX XXX","XXDXRXZZX XXXXL","XXDXRXZZX XXRXXXXRX","XXXXXX XXRXXXXRX","PRXVXXXXX XXRXXXXRX","XXP XXRXXXXRX","XXXXX XLXXXXX","XXDXXX PRXXXXX","DXXX XXXXXZXXXX","XXPX DXXXXXXXX","XHXXVX XXXXXRX","DXXX XXXXXXXXX XXXXXRX","DXXX XXXDXXZX XXXXXRX","XXPXRXX XXXXLX","XXPXRXX RXXXDXX XXXXDXXX","PRXXXXXXXXXXXXXXZXXXX PRXXXXXXXXXX","DXXX PRXXXXXXXXXXXXXXZXXXX PRXXXXXXXXXX"';  string  csvFile2='',csvFile3; system.debug(limits.getLimitHeapSize()); system.debug(limits.getHeapSize()); for(integer i=0;i&lt;3618;i++){ csvFile2+='\n'+csvFile; }   system.debug(1.0*limits.getHeapSize()); system.debug((1.0*limits.getHeapSize())/limits.getLimitHeapSize()*100);   with 3618 iteraction i get    Heap size 3001245.0 Which correspond to ~50% of the heap size limit in the Synchronous context   but if i go to 3619 iteractions in the for, i get the error "Apex heap size too large: 6001368".... which surprise me a lot...   I'm not getting out why this behaviour is occurring. Do you have any similar experience? There is a reason for this?  Seems more about limits on strings, in effect the string csvFile2 overcome 3M characters... but i dont know
99434
I am trying to update the debug log trace flag for my user but getting the following error     Having an active trace flag triggers debug logging. You have 259 MB of the maximum 250 MB of debug logs. Before you can edit trace flags, delete some debug logs.   I have removed all existing debug logs but it is still throwing the error.
91805
Short prestory. Typical problem: you have a lot of small string and you want to concatenate them all in one big string. The code like for(Integer i = 0; i &lt; smallStringsSize; i++){     bigString += smallStrings[i]; }  will solve this problem, but it has to produce a lot of unnecessary string objects that consume your heap. That is why Java has classes like StringBuilder and StringBuffer that allow you to solve this problem wihtout consuming too many memory. Thus, I decided to try to do my own StringBuilder in apex. String class has two methods that looked suitable for it:   and  . Here is the code that i used to test these methods against &quot;+=&quot; concatenantion (ready to execute in anonymous): List&lt;String&gt; chars = new List&lt;String&gt;(); List&lt;Integer&gt; charsCode = new List&lt;Integer&gt;(); Integer size = 10000; String chartToAdd = 'Ω'; for(Integer i = 0; i &lt; size; i++){     chars.add(chartToAdd);     charsCode.add(String.valueOf(chartToAdd).charAt(0)); } String joinS = '', forS = '', fromS = ''; Integer initialHeapSize = Limits.getHeapSize(); Integer initialCPUtime = Limits.getCpuTime(); System.debug('initialHeapSize ' + initialHeapSize); System.debug('initialCPUtime ' + initialCPUtime); joinS = String.join(chars, ''); Integer joinHeapSize = Limits.getHeapSize() - initialHeapSize; Integer joinCPUtime = Limits.getCpuTime() - initialCPUtime; System.debug('-------- String.join ---------'); System.debug('joinHeapSize ' + joinHeapSize); System.debug('joinCPUtime ' + joinCPUtime); for(Integer i = 0; i &lt; size; i++){     forS += chars[i]; } Integer forHeapSize = Limits.getHeapSize() - joinHeapSize - initialHeapSize; Integer forCPUtime = Limits.getCpuTime() - joinCPUtime - initialCPUtime; System.debug('-------- for each += ---------'); System.debug('forHeapSize ' + forHeapSize); System.debug('forCPUtime ' + forCPUtime); fromS = String.fromCharArray(charsCode); Integer fromHeapSize = Limits.getHeapSize() - forHeapSize - joinHeapSize - initialHeapSize; Integer fromCPUtime = Limits.getCpuTime() - forCPUtime - joinCPUtime - initialCPUtime; System.debug('-------- String.fromCharArray ---------'); System.debug('fromHeapSize ' + fromHeapSize); System.debug('fromCPUtime ' + fromCPUtime); System.debug(''); System.debug('-------- strings ----------'); System.debug('joins ' + joins); System.debug('fors ' + fors); System.debug('fromS ' + fromS);  The result of this code execution:  initialHeapSize 161166 initialCPUtime 242 -------- String.join --------- joinHeapSize 10016 joinCPUtime 17 -------- for each += --------- forHeapSize 10024 forCPUtime 48 -------- String.fromCharArray --------- fromHeapSize 10016 fromCPUtime 5  As you can see, the heap size for each approach is so close to 10 kilobytes, what is a size of the string of 10 000 characters (size) if each character takes only one byte. In case of creating 10 000 additional string objects, the heap size would be a bit more than 50 megabytes (10 000 * (1 + 10 000)/2). Taking into consideration the above,  Does not the '+=' concatenation create any additional string objects that devour your heap? Did not I miss anything? I am so amazed.  As you see, I use Omega 'Ω' as a chartToAdd. It has  the high-surrogate code point equals to 937, that needs at least 2 bytes to be encoded. How can 10 000 Omegas consume only 10 000 bytes of the heap size?  What weightful reasons Salesforce has that they do not to add the character data type? Any ideas?
31499
Does assigning an unused object reference to null in Java improve the garbage collection process in any measurable way?  My experience with Java (and C#) has taught me that is often counter intuitive to try and outsmart the virtual machine or JIT compiler, but I've seen co-workers use this method and I am curious if this is a good practice to pick up or one of those voodoo programming superstitions?
14370
I wanted to run a  query to return a large amount of data so I could export the lot to CSV. Problem is, the page is crashing my machine - the query triggers graphing, and the graphing code can't handle the data!  This screenshot of the insane graph was just barely obtained by another user:   I'm currently unable to modify the SQL as it kills the browser (latest Chrome on a Mac). EDIT Initially I thought it was simply due to the amount of data and perhaps that pagination would help, however the comments (thanks guys!) have uncovered that it was the graphs being drawn.
26577
I have a bunch of client point of sale (POS) systems that periodically send new sales data to one centralized database, which stores the data into one big database for report generation.  The client POS is based on PHPPOS, and I have implemented a module that uses the standard XML-RPC library to send sales data to the service. The server system is built on CodeIgniter, and uses the XML-RPC and XML-RPCS libraries for the webservice component. Whenever I send a lot of sales data (as little as 50 rows from the sales table, and individual rows from sales_items pertaining to each item within the sale) I get the following error:  Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 54 bytes)   128M is the default value in php.ini, but I assume that is a huge number to break. In fact, I have even tried setting this value to 1024M, and all it does is take a longer time to error out.  As for steps I've taken, I've tried disabling all processing on the server-side, and have rigged it to return a canned response regardless of the input. However, I believe the problem lies in the actual sending of the data. I've even tried disabling the maximum script execution time for PHP, and it still errors out.
8220
I know that the compiler will sometimes initialize memory with certain patterns such as 0xCD and 0xDD.  What I want to know is when and why this happens.  When  Is this specific to the compiler used?  Do malloc/new and free/delete work in the same way with regard to this?  Is it platform specific?  Will it occur on other operating systems, such as Linux or VxWorks?  Why  My understanding is this only occurs in Win32 debug configuration, and it is used to detect memory overruns and to help the compiler catch exceptions.  Can you give any practical examples as to how this initialization is useful?  I remember reading something (maybe in Code Complete 2) saying that it is good to initialize memory to a known pattern when allocating it, and certain patterns will trigger interrupts in Win32 which will result in exceptions showing in the debugger.  How portable is this?
70009
I ran into this error earlier today and requested that the extra logs be purged from the server. It was done but it still claims that there isn't any room. Is there a way to notify the server that it has more room than it thinks?
12219
I recently migrated an old server running MySQL to a new VPS running MariaDB 5.5. I don't have too much running on the server (just a few PHP sites) and free memory seems to be OK, but the DB keeps crashing--sometimes every few days, other times within a few hours.  I receive the following errors in the logs:  131231  1:43:04 [ERROR] mysqld: Out of memory (Needed 128917504 bytes) 131231  1:43:04 [ERROR] mysqld: Out of memory (Needed 96681984 bytes) 131231  1:43:04 [ERROR] mysqld: Out of memory (Needed 72499200 bytes) 131231  1:43:04 [ERROR] mysqld: Out of memory (Needed 54362112 bytes) 131231  1:43:04 InnoDB: The InnoDB memory heap is disabled 131231  1:43:04 InnoDB: Mutexes and rw_locks use GCC atomic builtins 131231  1:43:04 InnoDB: Compressed tables use zlib 1.2.3.4 131231  1:43:04 InnoDB: Using Linux native AIO 131231  1:43:04 InnoDB: Initializing buffer pool, size = 128.0M InnoDB: mmap(137756672 bytes) failed; errno 12 131231  1:43:04 InnoDB: Completed initialization of buffer pool 131231  1:43:04 InnoDB: Fatal error: cannot allocate memory for the buffer pool 131231  1:43:04 [ERROR] Plugin 'InnoDB' init function returned error. 131231  1:43:04 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 131231  1:43:04 [Note] Plugin 'FEEDBACK' is disabled. 131231  1:43:04 [ERROR] Unknown/unsupported storage engine: InnoDB 131231  1:43:04 [ERROR] Aborting  131231  1:43:04 [Note] /usr/sbin/mysqld: Shutdown complete   I've played with the my.cnf settings for InnoDB Heap, which does not seem to help. Here's the relevant portion:  innodb_buffer_pool_size = 128M innodb_log_buffer_size  = 8M innodb_file_per_table   = 1 innodb_open_files       = 400 innodb_io_capacity      = 400 innodb_flush_method     = O_DIRECT   I seem to have "plenty" of free RAM, and I have some swap available as well:  root@phoenix:~# free -m              total       used       free     shared    buffers     cached Mem:           994        923         71          0         19        417 -/+ buffers/cache:        486        508 Swap:         1023        131        892   How can I solve / troubleshoot this issue? I've scoured the interwebz for clues, but nothing has helped.
25857
My aim is to audit all queries on an instance, who's running them and from where, how many reads, in order to consolidate/migrate on another instance.   Figured out the best way is via Extended Events to .xel files, over 2 weeks, copy those and analyse on my well-specd desktop. Set up the capture with the minimal number of details (fields), so really can't make it smaller than this - aprox 12GB in 12 files. Used "Merge Extended Event Files" in SSMS to load the .xel files and process as detailed in   Is this really the best way to do it ? Is there a better tool to load the files ? It's really slow with SSMS 17.7 - example: with only 4 filters so far (before getting into grouping and aggregates) it took over 12h to get to crunch through 50% of the logs.
56349
I was on the latest version before this, 2.92. Everything was fine. I just updated, I clicked to import settings from 2.92 at the first run popup window, all ok. Then I loaded one of my scenes, everything ok. But now, if I try to render, the viewport remains grey, and once I try to revert the viewport to EEVE render, the program simply enters an infinite loop and freezes. The grey space in the middle of the image is the object &quot;rendered&quot; in &quot;Cycles&quot;.  And this below is what happens after I click another render method: menu stuck and mouse pointer rolling.  The same happens if I switch to another panel with a different render method. Update: this only happens when using GPU computer. If it's on CPU, it works. I have just tried with a brand new scene with the cube, where Cycles is set on CPU, and it worked. Once I tried to switch to GPU, again it rendered grey. Then when I reverted again to CPU, the program froze.
74444
Is it possible to prevent stack allocation of an object and only allow it to be instiated with 'new' on the heap?
19077
I developed an algorithm for a fairly hard problem in mathematics which is likely to need several months to finish. As I have limited resources only, I started this on my Ubuntu 12.04 (x86) laptop. Now I want to install some updates and actually restart the laptop (the "please reboot" message is just annoying).  Is there a way to save an entire process including its allocated memory for continuation beyond a reboot?  Here is some information about the process you might need. Please feel free to ask for further information if needed.   I called the process in a terminal with the command "./binary &gt; ./somefile &amp;" or "time ./binary > ./somefile &amp;", I cannot really remember. It's printing some debug information to std::cerr (not very often). It's currently using roughly 600.0 kiB and even though this will increase, it's unlikely to increase rapidly. the process runs with normal priority the kernel is 3.2.0-26-generic-pae, the cpu is an AMD, the operating system is Ubuntu 12.04 x86. it runs since 9 days and 14 hours (so too long to cancel it ;-) )
122116
We have a codebase that is several years old, and all the original developers are long gone.  It uses many, many threads, but with no apparent design or common architectural principles.  Every developer had his own style of multithreaded programming, so some threads communicate with one another using queues, some lock data with mutexes, some lock with semaphores, some use operating-system IPC mechanisms for intra-process communications.  There is no design documentation, and comments are sparse.  It's a mess, and it seems that whenever we try to refactor the code or add new functionality, we introduce deadlocks or other problems.  So, does anyone know of any tools or techniques that would help to analyze and document all the interactions between threads?  FWIW, the codebase is C++ on Linux, but I'd be interested to hear about tools for other environments.    Update  I appreciate the responses received so far, but I was hoping for something more sophisticated or systematic than advice that is essentially "add log messages, figure out what's going on, and fix it."  There are lots of tools out there for analyzing and documenting control-flow in single-threaded programs; is there nothing available for multi-threaded programs?    See also