We are running MatLab models for our investment fund. Some of the models take 5 days to run on a 3gig processor with 516megs DDR…I need a Cray or ??? I don’t even know where to start looking.
Thanks-
-Thomas
Bioengineering firms often have multiple Crays; I imagine if you asked one of them nicely they might lease you one.
On the other hand, I’m not sure how strong the bioengineering industry is in the Czech Republic…
You might wanna look into a multi-processor Opteron system, or maybe a Beowulf cluster if the models take that long.
“Those that understood him smiled at one another and shook their heads; but for mine own part, it was Greek to me” - Shakespeare
-Tcat
I doubt that a Cray is a solution. I don’t think MATLAB even runs on Crays. (Could be wrong about that).
There are two ways to approach the problem:
-
Apply more hardware,
-
Improve the algorithm.
You can go for approach (1) without touching Crays. Buy a faster computer, like DreadCthulhu suggested, or break the problem up and run the bits of it on many computers.
If you want help with approach (2), you’ll need to supply more details of what you’re actually doing.
I agree that the Beowulf cluster is the way to solve this problem. Cheaper than cray time and can be reutilized on your local network for other projects during offtime. Besides if over time it is not powerfull enough just add more omputers to the cluster.
Mmmmmmmmm…beowulf clusters…
Here’s a linkie for more information:
I was going to come in here and say Beowulf Cluster but I see I am not the first.
Mmmmm…tasty beowulf cluster
You might check to see if you’re invoking virtual memory at all. (512Meg isn’t what it used to be.) Once you start swapping, processor utilization drops to a few percent.
Parallel processing might help if your problem is nicely decomposable (and if your software supports it).
However you may just be running into a situation where your problem is intractable in terms of processing power and you have to switch from a deterministic solution to a heuristic algorithm. If your problem is growing in size exponentially, adding more processors linearly isn’t going to help.
Imagine a Beowulf cluster of Crays…
:: ducks brickbats ::
However, if you have a diagnostic rhetorism on your secondary hoobus, your nihilistic functions will probably just be sitting around painting, so you’ll want to occupy them with uncracked relostatory symptoms. If you do that, your elastic combinitory functions will become primary functions of your tertiary (not secondary) hoobus, which should, if managed properly, awaken Cthulhu.
Math makes my brain hurt.
:smack:
Ah, but then your chip’s tricuspid valves would be operating out-of-synch with the diastolic frequency, leading to highly suboptimal throughput rates and the potential of double-processing. While VLIW architectures can supersaturate to compensate for this net reordering effect, your pipeline will still be orthogonal to your prefetch cache, causing nearly-random branch prediction and thus destroying your efficiency.
(This is probably too much fun for GQ.)
My husband was drooling over this yesterday:
http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=3032445399&category=1484
It’s only $4500 but it ends in 2 minutes. better hurry!
Buy a Power Mac G4. Then you can OWN the damn supercomputer, intstead of just RENTING it.
I don’t know what the details are of the problem you are working on, but parallel processing may not help. Matleb itself does NOT support parallel processing (e.g., on my dual-processor machine, one serial Matlab job can oonly achieve 50% CPU utilization).
There are some approaches to providing parallel functionality to Matlab. This is particularly easy if the problem is “embarrassingly parallel”, i.e., you are performing many repititions of a calculation that do not depend on each other. Some statistical resampling schemes fits this well, but many other algorithms do not.
Otherwise, you are in for a lot more work. Check out this link for a survey of different parallel Matlab approaches.
If you’re problem does not need any specific functionality of Matlab, you could consider moving to another environment that supports parallel processing.
Guys – it’s just not that complicated what I said. Obviously I don’t know the details of Tomcat’s problem, but maybe he’s trying to allocate money to a bunch of different investments, each one with its own management cost, startup cost, and maybe minimum and maximum investment amounts. The key is that the problem has a lot of possible combinations.
Suppose you have three funds, A, B, and C. You can invest in A, B, C, AB, BC, CA, ABC or none of the above. 2N choices. The number of possible combinations grows exponentially. Now suppose you have 100 possible funds to invest in. That gives you 2100 possible combinations. That’s a really big number. Even if your problem can be solved using parallel processing, you can’t keep up with this kind of problem by adding new processors because, in this case, every time you added a new fund you’d have to double the existing number of processors in order to keep the solve time constant.
That solution would make Dell and Microsoft and Intel very happy, but the rent on a universe big enough to hold all the computers you’d eventually need would be pretty high. Not to mention the electric bill. So, what do you do? Well, you effectively give up on getting the one best solution, and decide that you’ll accept one that’s good enough. You use heuristic (rule-of-thumb) tricks to avoid looking at combinations that don’t look like they will be profitable. You try to guess which combinations to try first, based on your knowledge of the domain. And you try to figure out ways of telling when the solution you have is close enough to the “real” solution so that you can stop looking.
Thanks folks. I’ll have the mathematicians read this and see if it helps. Finangle could be right though. We track 50 indices and data sets with daily data going back over 10 years. The combinations are pretty scary. And soon we are going to be modelling intra-day tick numbers (1, 5, 10, 15, 30 minute and hourly) on those same indices. And then we might start modelling the underlying stocks within the indices. We’re talkin’ lots of numbers, m’kay? And these things go beyond common regression and trending analysis. We’re into Fuzzy logic systems and might even try genetic programming soon.
-Tcat
I didn’t see any math in Finagle’s comments. Just 
computer science.
Which is why I am constantly pointing out to people that I am NOT a computer science major, notnotnot.
Oh, I could probably learn it if I wanted to.
I just don’t want to.