For an application I'm working on, we have to make a call to a database where everything is tallied up and then displayed to the user. The problem presents itself in load time. Currently, it takes about 8 minutes to get all of the data collected, counted, and sorted which is WAY too long.  Other team members are working on speeding it up, however if it isn't possible because of the sheer amount of data, we'll have to show the user that their data is loading, it's just taking a long time and they can come back soon.  What is the best way to do this? Should text appear after a certain amount of time telling the user what is happening? Currently, we have a loading spinner that looks like the one below (converted to png for UX.SE), but is that sufficient? I feel like the user would think the application stalled.
120812
I'm working on a site where I am being asked to implement a loading animation on page loads. Essentially, the site will display a loading animation as soon as possible, which will go away once all of the assets (images, essentially) have been loaded.   I'm worried that this will actually reduce usability, but am very interested in any data on this. I want to know the "right" answer.  Essentially I have two feelings:   If images further down the page are still loading, I as a user, don't care. I want to start consuming any content as soon as it's available. From this perspective, hiding the site until everything is loaded would actually reduce perceived performance.  The browser already gives a good indication that the site is still loading. Users are at least subconsciously aware of this and are used to some things appearing after the initial render.    I've seen this sort of thing often on single page javascript applications where I think it makes a lot more sense. The user has an initial loading page, and afterwards the application is ready to go with all assets loaded. But, seeing a loading image every time you click a link feels a bit off to me.   I am very interested in opinions or data from people more knowledgeable and experienced in UX than myself.
90023
this is my first time to try this "Defer Parsing" thing, and I don't really have an idea how to do it.    Using PageSpeed tool, I found out that one of the reason why my page is loading too slow (currently 92/100) are the scripts. And the PageSpeed suggested that I need to Defer Parsing all those scripts.  Is there anyone that can help me with this. Thanks.
119639
Recently I've noticed a new trend in user interfaces. Facebook and Twitter do this, and I've come across it on other random, less famous sites. However, to keep things simple, I'll just use Facebook as an example.  When you first log into Facebook, you are presented with a "News Feed" that has a certain amount of posts from your friends. The feed extends down past the bottom of the viewport.  Then, when you scroll down, to see the feeds "below the fold", at some point as you move downward, it loads in more items. Apparently, this is called "infinite scrolling".  On my computer, this often causes my position in the feed to jump while I'm in the action of scrolling, which is really annoying.  What's particularly egregious about this UI concept is that there is footer information at the bottom of the page. I advertise on Facebook, and use the link at the bottom of the page to get to the advertisement management interface.  Because of this incremental load feature I'm describing, I often scroll down to the bottom to get to the link I want, when I reach the bottom of the page, it will suddenly load in new data, pushing the footer down again, and I have to scroll down again.  This whole thing seems completely pointless to me. What's the upside? A Facebook page is near a megabyte of data, so saving the few extra kilobytes of feed data for when the user scrolls down is just not worth it.  What's wrong with a link that just says "show more"?  Since this additional data retrieval thing seems to happen as soon as I start scrolling even a little anyway, why not just load that data in the first place?  To me this minor annoyance reeks of some engineers concept of a "neat trick" that does not have any actual appreciable upside.  Or am I wrong about that? Is there an advantage that I'm not seeing? Am I the only one who thinks it's stupid? Do some people like it?
40526
In my application, I have pages that show lists of some objects, for example "People" or "Contracts".  From these pages, the user may create a new object.  Often these objects only have a handful of properties, maybe 4 or 5.  My question is about the best approach for the forms used for creation of new objects.  Sometimes I have used a separate page, especially when the number of properties is greater than 5-6.  Sometimes I have used a modal dialog.  I have even seen some applications that insert an empty row into a table and you fill-in the values.  (I will not be doing that!)  Any UX-preferences or suggestions for what approach to take?  Also- is it important to be consistent between different types of objects?  Or is a mix of approaches OK?  The technology of my application (Ember.js, single-page app) allows me to navigate to a different page instantly, so any timing difference between approaches is not a factor.  In fact, the animation of the Bootstrap dialog takes longer than the page transition.  The application is a back-office administrative system for insurance companies.
104158
I wanted to get some UI design advice. I am not sure this is the proper forum to ask this question so here goes.  My situation is I have a table that has 51 bit values to hold the values of statutes.     In production, this table would have over 10,000 records for the different LocationID's.  The UI for this in its pure database for would have a page with the dozens of text boxes.  Is there a better, cleaner way to do this?  Any recommendation or links to how others have handled this situation.  Please feel free to post any additional questions if further clarification is needed.
36453
I'm writing an import utility that is using phone numbers as a unique key within the import.  I need to check that the phone number does not already exist in my DB. The problem is that   phone numbers in the DB could have things like dashes and parenthesis and possibly other things. I wrote a function to remove these things, the problem is that it is slow and with thousands of records in my DB and thousands of records to import at once, this process can be unacceptably slow. I've already made the phone number column an index.  I tried using the script from this post:   But that didn't speed it up any.  Is there a faster way to remove non-numeric characters? Something that can perform well when 10,000 to 100,000 records have to be compared.  Whatever is done needs to perform fast.  Update Given what people responded with, I think I'm going to have to clean the fields before I run the import utility.   To answer the question of what I'm writing the import utility in, it is a C# app. I'm comparing BIGINT to BIGINT now, with no need to alter DB data and I'm still taking a performance hit with a very small set of data (about 2000 records).   Could comparing BIGINT to BIGINT be slowing things down?  I've optimized the code side of my app as much as I can (removed regexes, removed unneccessary DB calls). Although I can't isolate SQL as the source of the problem anymore, I still feel like it is.
39594
I'm trying to find the best way to save the state of a simple application. From a DB point-of-view there are 4/5 tables with date fields and relationships off course.  Because the app is simple, and I want the user to have the option of moving the data around (usb pen, dropbox, etc), I wanted to put all data in a single file.  What is the best way/lib to do this?  XML usually is the best format for this (readability &amp; openness), but I haven't found any great lib for this without doing SAX/DOM.
55185
An interesting addition to the data dump would be the daily/hourly/whatever view counts for the websites. This could be used to see what happened to SO when proposal X came out, what time of day is the most active (if it was hourly), what day of the week is the most active, activity to question ratio, and many other things.   Before you say "What about Users.Views and Posts.ViewCount?", they aren't usable since they only list the total, not the breakdown of when it happened. The only statistics you could draw are monthly, which is useless.  If your worried about size, then your exaggerating what I'm asking. Daily is tiny, only 7 rows for an entire week. Hourly is a bit bigger, 168 rows per week. Considering other tables are millions of rows long, this won't affect size at all considering the growth.
114911
Problems   Seeing this wait increase and very high, which it never shows up as a wait. Users don't get back to the database when they make requests - it takes a while before they land on the database and when they do, they get an immediate resolution to query/insert/etc. It really appears there's an issue with the users getting confirmed before querying - this is just a guess based on some reading.  Microsoft provides nothing on this wait that I can find. The normal volume for users dropped 99% when this wait took off. Developers seeing "you are not authorized to access microsoft iis" on their end - but we've checked disk space on all servers, which is fine.   What we've done:   Restarted all servers - SQL and web.   CPU, memory, disk all seem fine on SQL Server. This wait continues to increase. Error log doesn't show any abnormal behavior.   Overview of issue: people will run a query on the website, we don't see it on the back-end until much later, when I do see it, it runs immediately and they get their result.  There seems to be a delay from the intermediate server to the database server, possibly related to authentication (a guess based on this wait).  It's possible this wait is completely unrelated, but it continues to increase and I've never seen this.
97239
OK, here's the situation. Someone posted an  on SO.  There were 2 main answers.   One was stock "you can use this standard data structure - see details on Wiki". Basically, a priority queue.  One was mine, which had a somewhat-original (at least, I don't recall ever reading an official moniker for it) data structure. Actually, two of them, one for each possible domain of data distribution.    BOTH algorithms were faster on average (mods AND look-ups) than the standard Priority Queue when applied to that user's problem domain. The proof, aside from fairly simple O(n) analysis, is that as of today, the OP commented on my post that my data structure was the best he could use and accepted my answer. However, at the time of posting, a couple of people down-voted it initially, with incorrect excuses like "algorithm does not work" (based on incorrectly reading my answer), "Wiki data structure is better" (without reading my O(n) analysis or comprehending it). First down-vote was from a competing Wiki question. Then it seemed to be lemming-like down-voting, including at least 2 serial down-voters that were since cleansed by SO team as per my report.   So, since most of the current down-votes if not all are based on people being obviously technically wrong as acknowledged by original poster, is there some sort of appeals process on SO whereby a technically competent moderator/superuser can review the Q, determine whether the people who down-voted also posted incorrect comments as reasons for down-voting, and fix this?  Please note that this is mostly a question of fairness - at my rep level, the current "-6" rep from the Q are not really that big of a deal (and the main problem was the opportunity cost of people not voting my A up when the thread was active due to seeing negative score and a long thread of comment criticisms - which I can't get back anymore since the thread is no longer active).  And yeah, I'm whining :) But it was a behavior of a couple of people that was grossly unfair and totally contrary to the spirit of SO.
123896
Been doing tests in sandbox in which I've created about a million accountshares that I need to get rid of for testing.   I'm trying data loader in bulk api mode to export than delete but I keep getting:     InvalidBatch : Failed to process query: QUERY_TIMEOUT: Your query request was running for too long. Trying again later.;   Not sure if I should be waiting longer? Any other options or dataloader configurations I should be looking at?  EDIT  I've been trying with batch apex jobs yet still having issues.   Script  string str='select id from AccountShare where lastmodifiedbyid =\'0050P000007Kfy2\' limit 10000'; sharedeletebatch bdt=new sharedeletebatch(str); Database.executeBatch(bdt); system.debug(bdt);   Batch  global class sharedeletebatch Implements Database.batchable&lt;sobject&gt; {       global final string query;      global sharedeletebatch(string q){            query=q;      }       global Database.QueryLocator start(Database.BatchableContext BC){        return Database.getQueryLocator(query);      }      global  void execute(Database.BatchableContext BC,List&lt;SObject&gt; scope){          delete scope;      }     global void finish(Database.BatchableContext BC){     }   }      First error: [REQUEST_RUNNING_TOO_LONG] Your request was running for too long, and has been stopped.
54938
I have a database, that due to some silly code issues has grown hugely. It is now sitting at 24GB, most of it is very unnecessary logging information (system generated debug information).  My server lives on a cloud hosted server. I am now sitting with 2 issues because of the file size:   I pay for storage. I am happy to pay for my core business, but paying for silly data seems....silly My backups are now huge also. I do nightly full backups, and then ship them off to an FTP server. This process is taking longer and longer.   Given my issues, is shrinking so bad? I will rebuild all my indexes. I did a dry run, and brought my database down to 6GB.  This guy (who I have always trusted as my SQL guru), says it's bad....mkay
61917
I want a splash screen to show while the application is loading. I have a form with a system tray control tied to it. I want the splash screen to display while this form loads, which takes a bit of time since it's accessing a web service API to populate some drop-downs. I also want to do some basic testing for dependencies before loading (that is, the web service is available, the configuration file is readable).  As each phase of the startup goes, I want to update the splash screen with progress.  I have been reading a lot on threading, but I am getting lost on where this should be controlled from (the main() method?). I am also missing how Application.Run() works, is this where the threads for this should be created from? Now, if the form with the system tray control is the "living" form, should the splash come from there? Wouldn't it not load until the form is completed anyway?  I'm not looking for a code handout, more of an algorithm/approach so I can figure this out once and for all :)
20338
I've noticed this phenomenon a few times before: find a question that was posted 10 minutes ago, look at the questioner's profile.  Last seen 20 minutes ago.  Does updating last seen happen asynchronously with posting a question?  How long might the lag be?  I'm reluctant to , since the phenomenon is probably short lived, but I just noticed it there.  Here are screen images as of this writing...  On the question:        In the profile, clicked a second later:
10779
For application, which has few (5-10) settings at the moment but will have a lot more (up to 100) in the future which would be better approach:   to add new columns to settings table every time new setting is introduced to have settings table as key/value storage   Consider that application might have millions of instances (which all run on the same db and use the same table for settings), DB might be sharded. DB is relational, i.e. either MySQL or T-SQL.  As a developer I would prefer the 2nd variant, so I could scale the application without changing the DB schema and add/remove settings at will. As far as I understand having several million records in a single table shouldn't be an issue if the index is clustered on application instance. Are there any drawbacks that I'm not aware of?  And what about 1st variant? Are there any big benefits? And what about the number of columns: is there any theoretical limit of columns that table can have? What happened if I had a table with 1000 (10 000, 1000 000) columns? Would that be slow?
85999
What are the technologies and programming decisions that make Google able to serve a query so fast?   Every time I search something (one of the several times per day) it always amazes me how they serve the results in near or less than 1 second time. What sort of configuration and algorithms could they have in place that accomplishes this?  Side note: It is kind of overwhelming thinking that even if I was to put a desktop application and use it on my machine probably would not be half as fast as Google. Keep on learning I say.    Here are some of the great answers and pointers provided:      Hardware - cluster farms and massive number of cheap computers Caching and Load Balancing
108241
I have a website the seems to load very slowly. When I run a speed test on it I see there seems to be a 6 secound gap before the HTML loads. Images and JS scripts load very fast after that point.  You can see in the pic below the yellow 'wait time' bars:    This seems to be consistant no matter what the HTML content on the page is.  This site is using a CMS (ModX Revo) so HTML is actually stored in an SQL database and brought forth by PHP, but I've never experienced this issue before.  Would anyone know what is causing this and how I could speed it up?