My mechanic says he's not able to fix my car by phone. To be fair, I would even allow him to give it a look from the outside, but that's all; nothing of opening the thing and seeing its guts, are you crazy or what. I bet the bastard could fix it by ear, but wants to fiddle with the poor thing just to bother and charge me twice. I have the same problem with physicians. And fridge repair staff. Etc.
Computer science guys know better. We insist on debugging something by ye olde try-and-fail method, the same method employed to debug Eliza, Ultrics, hell, you know where that Tron movie came from. But it's not 1970 anymore, which means that your IDE of choice surely offers more reasonable alternatives. Assuming a standard project 99% composed of open source frameworks linked together, the libraries sources can be attached:
This allows debugging and navigation through the java class files, handy when investigating ways to initialize Velocity from the classpath or understand the spring test framework. As a bonus, I use to include the version number in the source zip file to keep track of each library.
The same can be done with the JDK (as a matter of fact, it is almost mandatory if you want to do something non-trivial). Open Window->Preferences->Java->Installed JREs, and make sure you have selected a JDK, not a JRE:
The main difference is that eclipse will automatically link the included src.zip file of your JDK, so you will also be able to debug inside your JVM classes:
With closed source this is not an option. Still, you could reverse-engineer code (at least, the class that is throwing that annoying NPE), but that would probably go against your EULA. It is illegal. I am not advising you to do so. I Could Be. But I Am Not.
The testsuite should test 99% of reported bugs, but there is always that one that does not throw a stack trace and must be debugged inside the app server. For those, God (ok, and sun) invented remote debugging with JDWP. For weblogic, create a debugWeblogic.cmd file in the same folder as startWeblogic.cmd:
set JAVA_OPTIONS=-Xdebug -Xrunjdwp:transport=dt_socket,address=1044,server=y,suspend=n .\startWeblogic.cmd
Now, create a remote debugging config inside eclipse:
I use to share this configuration: click the common tab, select shared config, save it into a dedicated folder inside your project (named eclipse-conf or similar), and commit it to your repository. Now everybody in your team can debug remotely their own local server, if needed.
I honestly do not believe in current state-of-the-art JSP debuggers. The generated java file (you know that jsp translates to a servlet that is compiled, right?) is not that complicated to read, so why not configure your server to write the servlet to disk and not remove it after compilation? (weblogic and jetty sure allow this) After a little visual inspection you could even debug inside, though I have never gone that far.
About profiling: premature optimization is the root of all evil. Having said that, when profiling arrives you better measure, don't guess. In my experience profilers are good for fixing memory leaks and could also be used for performance but most of the time with a bit of common sense you will be alright.
Database access is measured in milliseconds while memory access is done in nanoseconds. That means that if you focus your effort in removing redundant database access you are being 10^6 times more productive than improving memory consumption or for loops. I will repeat that again: memory is one million times faster than disk.
Get a list of the most frequent database accesses: revise all lazy relationships, eliminate those not needed and create all indexes required by queries. For this one, SQL EXPLAIN PLAN is a must: it will output the indexes used by a query, because MAN YOU DON'T KNOW IF YOUR QUERY IS USING THEM unless your database tells you so.
Consider using a web caching framework like oscache. Database caches are useful, but what really hits the problem on the head is caching entire web pages. This framework completely saved an online newspaper that called me to fix their lame pages-per-second ratio.
There are not black boxes anymore: to really know what's happening you need to see. Try-and-fail methods may be good to learn how to park your car, not for laparoscopic surgery.