AspenPLUS Workstation

[Natural ChemE]

I'd like to build a workstation for one of my CAD programs called AspenPLUS. That it's purpose-built for AspenPLUS imposes a few important constraints.

I have a bunch of questions which I suspect are pretty tough to answer, but any help on any of them would be greatly appreciated.

Constraints
1. It has to be on Windows 7 due to a whole host of compatibility issues which are notoriously difficult to deal with. I'll be using Windows 7 Enterprise x64.¹

2. AspenPLUS will use one and only one core for its runs.²

Questions
1. I assume that my workstation will work at about [the core frequency] times [the core multiplier] in operations per second. Is this accurate?

2. If I get a CPU with a lot of cache, how would it affect program operation? Would the computer automatically use the cache, or would it not know to do so and the extra cache go to waste? Would the 30MB of cache on new, high-end Xeons be very useful?

3. True or false: I want one core per running Aspen simulation plus one core for the OS and other background programs.

4. Since the computer will run nothing but Aspen and the OS, I doubt that it would require more than 28GB of harddrive space. If I get 64GB of RAM, then I can dedicate 32GB to a RAM disk and run the whole thing off of DDR3. Would this be helpful, or would it actually cause a bottleneck with communications to the RAM (since RAM's now doing double duty as memeory and storage) which would slow things down?

5. Not planning on doing much in the way of intensive I/O, and it wouldn't be a problem to backup my data every day. Would RAID still be useful, or can I just give it a pass?

6. Would there be any point in splurging on a high-end graphics card? (Aspen has pretty minimalistic graphics.)

7. How does one choose the optimal type of RAM? (The Xeon E7-8870 supports up to 1333MHz, yet the linked page notes that its max speed is 1066MHz. Does this imply that the 1066 is faster than the 1333 for it, or merely that the 1333 enjoys no advantage?)

8. Anyone happen to know what the odds are that a better Xeon will be coming out soon? And if it'd be much more expensive at introduction or not?

9. Say that I'm at the workstation (as opposed to remoting into it) and I want three screens. What sort of equipment would be necessary for this beyond the screens themselves? Just a second graphics card?

Xeon Comparison
Xeon X5687: 4 cores; 3.6GHz; 27x multiplier; 4x256kB L2 cache; 12MB L3 cache; 2x6.4GT/s bus. (1663USD)
Xeon E7-8870: 10 cores; 2.4GHz; 18x multiplier; 10x256kB L2 cache; 30MB L3 cache; 4x6.4GT/s bus. (4616 USD)

Since 27x3.6GHz is 2.25 times higher than 18x2.4GHz, the much cheaper X5687 would be the ticket for this purpose?

Weird observation: The X5687 supports up to 288GB RAM while the far more expensive and more-cored Xeon E7-8870 only supports 32GB RAM. Windows 7 itself is limited to 192GB RAM.

See: List of Xeon processors for numbers. I suggest using Ctrl+F to find specific Xeon's.

Notes
Like a cluster, I'd probably want to remote into it sometimes, and I may end up offering use of it to my coworkers. This is a minor consideration, but if it brings up any unique issues I'd be interested in knowing.

Footnotes
¹I'd be open to the idea of using some recent version of Windows Server if there's any expected gain to be had there, but unless there's motivation to move in that direction, I won't mess with it.

²So far as I know, anyway. On my dual-core laptop, it uses up to 50% CPU time when processing. It occasionally hits 51% for a moment but I suspect that these are estimation errors.

[kidjan]

1. I assume that my workstation will work at about [the core frequency] times [the core multiplier] in operations per second. Is this accurate?

No, this is wrong. The multiplier is what dictates the final frequency of the processor. See below for more detail.

2. If I get a CPU with a lot of cache, how would it affect program operation? Would the computer automatically use the cache, or would it not know to do so and the extra cache go to waste? Would the 30MB of cache on new, high-end Xeons be very useful?

In general, more cache is better. And yes, the computer will automatically make use of the cache as effectively as possible. And in theory, the 30 MB of cache would be better than the 12 MB since it's shared between all of the cores, but in practice it would really matter if the program could use the extra cache. If it's constrained in the L1/L2 caches, then the extra L3 might not make a huge difference.

3. True or false: I want one core per running Aspen simulation plus one core for the OS and other background programs.

True, although this will happen with no effort on your part. The OS will schedule stuff to run like this automatically.

4. Since the computer will run nothing but Aspen and the OS, I doubt that it would require more than 28GB of harddrive space. If I get 64GB of RAM, then I can dedicate 32GB to a RAM disk and run the whole thing off of DDR3. Would this be helpful, or would it actually cause a bottleneck with communications to the RAM (since RAM's now doing double duty as memeory and storage) which would slow things down?

You could do this, but it's not worth the effort. Just get a fast SSD.

5. Not planning on doing much in the way of intensive I/O, and it wouldn't be a problem to backup my data every day. Would RAID still be useful, or can I just give it a pass?

If you're not doing intensive I/O, then RAID won't help (in fact, it'll cost money and be a point of failure). Just get a fast SSD.

6. Would there be any point in splurging on a high-end graphics card? (Aspen has pretty minimalistic graphics.)

Beyond paying more on your electric bill, no. Not unless Aspen somehow leverages the GPU, which seems unlikely since it's still single-threaded.

7. How does one choose the optimal type of RAM? (The Xeon E7-8870 supports up to 1333MHz, yet the linked page notes that its max speed is 1066MHz. Does this imply that the 1066 is faster than the 1333 for it, or merely that the 1333 enjoys no advantage?)

See this; it might shed some light on what sort of performance gains you can expect from faster memory. But you might as well go for DDR-1600 since the cost difference is tiny.

8. Anyone happen to know what the odds are that a better Xeon will be coming out soon? And if it'd be much more expensive at introduction or not?

Ivy bridge will be coming out soon, which is purportedly 20% faster, but it's probably at the end of this year or even beginning of next year.

9. Say that I'm at the workstation (as opposed to remoting into it) and I want three screens. What sort of equipment would be necessary for this beyond the screens themselves? Just a second graphics card?

Get an ATI card with eyefinity; most of these cards can drive three or more monitors.

Xeon Comparison
Xeon X5687: 4 cores; 3.6GHz; 27x multiplier; 4x256kB L2 cache; 12MB L3 cache; 2x6.4GT/s bus. (1663USD)
Xeon E7-8870: 10 cores; 2.4GHz; 18x multiplier; 10x256kB L2 cache; 30MB L3 cache; 4x6.4GT/s bus. (4616 USD)

I'd actually consider the i7-2600k processor. It's way cheaper, close to identical to the Xeon X5687, and once you overclock it (which it was designed to do, because it has an unlocked multiplier), it'll be faster. Most people get ~4.2-4.5 GHz out of a core i7-2600k with little or no effort.

Since 27x3.6GHz is 2.25 times higher than 18x2.4GHz, the much cheaper X5687 would be the ticket for this purpose?

No, no--this is wrong. So the x5687 is clocked at 3.6 GHz, and the E7-8870 is clocked at 2.4 GHz. The multiplier doesn't factor into it like that--note that when you divide the clock speeds by the multiplier, both are 133 MHz. The multiplier basically determines the final clock speed of the CPU.

But comparing these two chips, the X5687 is going to be significantly faster than the E7-8870 because the individual cores are clocked much faster. If the workload was easily split up onto separate threads, the 8870 would be a better option since you could distribute the load across all those cores, but for single-threaded performance the 5687 is the clear choice.

[Natural ChemE]

Kidjan,

Awesome, thanks man. Especially for setting me straight on the whole clock multiplier issue.

I was hoping to get a Xeon as opposed to a Core i7 since the i7's can't handle ECC RAM, and I'll be doing some pretty long computational stuff. I did some reading up about the Xeon's, and it seems that the 56xx series uses SATA II and has some other older specs on it. The Xeon E series seems to be the newer version, with E-3xxx and E-7xxx already out, but the E-5xxx is currently held back (supposedly to help retailers clear 56xx's before obsoleting them). The Xeon E-5xxx should replace the Xeon 56xx series and, I'm assuming, have SATA III and the other newer qualities. I may end up holding out and waiting for this E-5xxx series to come out.

To note it, the new i7's should be coming out next month. Both the new i7's and the E-5xxx series should use a socket called "LGA 2011" (Wikipedia).

So this brings me to the next question - motherboards. I don't know much about how to intelligently select a high-performance motherboard beyond just making sure that it has the right socket(s) for the processors and can handle the desired amount of memory. What else should I look for? And are there motherboards for this "LGA 2011" yet? Is "LGA 2011" even an official name or just a placeholder?

Edit: Checked Wikipedia and found an article on the LGA 2011 socket, so my main question would be, "How do I figure out which motherboard would be best for my workstation?"

Edit 2: Looks like the LGA 2011 motherboards are already produced even though there aren't any chips that go into 'em yet.

[kidjan]

There's a bunch of LGA 1155 boards that support ECC, so you should be able to get an i7-2600k with ECC without too much trouble. The motherboards are a bit more pricy, of course, but that's to be expected for a feature like ECC.

As far as motherboard features go, I'd definitely opt for one with SATA III (the 6.0 gbps standard), since many SSDs can easily be bottlenecked by SATA II. Gigabit NICs are necessary, and two is better. USB3.0 is preferable, for faster external drive and thumbdrive access. PCI-e is also not something I'd do without. Decent electrolytics is also a plus (solid state caps throughout--nice reliability perk). I'd probably go for this motherboard, personally. Again, I think you'd actually get the best performance from an overclocked i7-2600k, since it has an unlocked multiplier, but ymmv.

Ivy Bridge looks quite nice, but I playing the "wait and see" game with hardware can go on forever. I tend to try and make the best decision "now" and not try to time the market. Guess you have to decide if you need it "now," or if it can wait.

[Natural ChemE]

I was under the impression that both the processor and the motherboard needed to support ECC, and if either didn't, then the ECC code couldn't be used (even if the sticks themselves could be used as normal RAM). Is it enough to use an ECC-supporting motherboard with a non-ECC-supporting processor?

And you're probably right about the wait-and-see bit. It's wasting too much time already.

[kidjan]

You're right--it would require processor support, since the Core architecture has the memory controller on-die. From what I saw, the i7 does not support ECC.

So if that's a requirement, you'll have to go Xeon. That said, you could still go socket 1155; I'd personally buy the E3-1270 and pocket the extra cash, since AspenPLUS is single-threaded. There are slightly faster E3-xxxx processors, but you'll pay out the nose for another 50-100 MHz--not worth the cash.

Also, wasn't paying close attention, but the Xeon X5687 is based on the older Nehalem microarchitecture, whereas the other two processors (Xeon E7-8870 and E3-1270) are both Sandy Bridge processors. I'd definitely opt for the newer architecture, and I'd definitely opt for one of the E3 variants, since their single-core clock speeds are much higher. The X5687 would likely be either about the same or slightly slower than the E3, and it's going to cost you about $1300 more.

[kidjan]

There's a combo right now on that Asus motherboard and the E3-1270; $530 for both. Hard to beat that. Pair it with 16 GB of memory for ~$160, a very nice SSD (go Intel 320; not the fastest, but the most reliable, and that's more important), a decent video card, and you've got yourself a pretty ridiculous workstation.

Those other chips are temping, but you're not going to see a significant performance improvement despite dropping $1000s more. Might as well spend the money on 30" monitors or something else flashy and awesome.

[Natural ChemE]

Kidjan,

Thanks for all of the help. I ended up selecting:
-Intel i7-3930k (Intel, NewEgg),
-ASUS P9X79 Deluxe (NewEgg),
-Corsair Vengeance 16GB (4x4GB) DDR3 1866 (x2, for 32GB total) (NewEgg),
-SeaSonic Platinum-1000 (NewEgg),
-Radeon HD 6970 (NewEgg),
-Crucial m4 256gb (NewEgg),
-ASUS VH242H screen (x3) (NewEgg),
-Corsair H100 water cooler (NewEgg),
along with a good case, BluRay burners, internal USB hubs for dongles, etc. This thing takes up enough space that I just decided to use it as my new desktop instead of remoting in. If anything, the three screens will make remoting into clusters and such so much more convenient; I haven't had more than two screens to a box before.

EDIT: Long list of questions deleted.

[kidjan]

Sorry, I did see your questions and was going to respond. To make a long story short, before you do anything fancy with a RAM disk, I'd do some actual testing to see if it makes any difference whatsoever. If it helps, then go for the extra RAM. If it doesn't, save your money. If disk IO isn't a bottleneck, then the ram disk is basically extra risk for no return and I'd skip it.

Other than that, looks like a pretty sick machine. And I take it you're planning on overclocking with that water cooler? You should be able to hit ~4 to 4.5 GHz without too much trouble.

[Natural ChemE]

Kidjan,

Thanks for all of the help with it. I ended up asking many of the retailers those questions and, to my (naive?) surprise, I got different answers.

For instance I was trying to figure out if the computer could take 64GB without worry since Intel's specs show the max memory for SBe processors as 32GB yet retailers sell 64GB. From Intel:

We have received your request. Please bear in mind that the motherboards may be supporting 64GB memory and different speeds and voltages. However, the memory is not[sic] longer controlled by the motherboard.

Since the memory controller is integrated on the Processor, we need to follow certain specifications on size, speed and voltage. In this case, the maximum memory size supported for Intel® Core™ i7-3930K Processor is 32GB.

Working over these specifications may affect the performance and also may damage the Processor.

I also asked in several other emails and chats, all with the same response.

From G Skill (who is currently the only supplier of 8x8GB kits on NewEgg):

With the combination of a high performance motherboard such as ASUS, and G.Skill high performance memory, there is no issue of performance loss.

The Intel statement simply means that the first 32GB (quad channel) is always active, whereas the second quad channel will only be in effect should your system attemps[sic] to access or needs it. This is similar to CPU stepping, so it can save power and increase efficiency when not in use.

As far as performance, there is no downfall, which is why we have DDR3-2133+ and such exclusive high performance kits that only G.Skill can offer.

The RipJaws Z series is designed specifically for the X79 platform, so any of the 64GB kits are guaranteed to operate to specification.

I got G Skill’s email late and will be sticking with 32GB for now, especially since tests show that the CPU is the bottleneck when the “working space” where most of the reads and writes happen is outsourced to a 2GB RAM disk.

I did manage to overclock it fairly well, getting up to about 5.0GHz without much trouble. Thankfully the P9X79 motherboard offers a great deal of support for overclocking which made it easy to do. I cranked it back down to about 4.75GHz as to avoid unnecessary deterioration on the i7-3930k. , so I think I’m good on CPU longevity (this guide warned that would result in increased deterioration rates). It actually runs pretty cool – slightly below human body temperature!

You know what’s weird, though? Before overclocking, the i7-3930k pulls back a Windows Experience Index (WEI) of 7.8. After overclocking it still pulls back a 7.8. While I realize that the WEI isn’t the normal kind of benchmarking tool, I’m confused as to why cranking it up a GHz over its stock turbo speed of 3.8GHz doesn’t result in even a 0.1 increase in the WEI. It definitely seems to crunch numbers faster.

I also cranked the Process Priority up to High (and occasionally to Realtime). High seems like the best option; Realtime will crash things that try to read the core’s usage, like Windows Task Manager, until the CAD finishes its run.

The last thing going on is with VT-d. I’m planning on making a few virtual boxes with AspenPLUS running in them, then coordinating them from the main OS space to perform specific runs. It’s my understanding that VT-d helps computational programs in virtual boxes, but I’m pretty unclear on the specifics.

The i7-3930k and the i7-3960x both suffer from an error in the first stepping, C1, which caused them to have their VT-d features disabled. According to Intel a BIOS update should allow for VT-d to work, and ASUS has released such a BIOS update for the P9X79 Deluxe that I have. I flashed it successfully, but none of my software lists VT-d in list of chip features even now. I don’t know if Intel was wrong, or if the BIOS update is buggy, or if my chip’s buggy, or what.

Anyway, save for resolving the VT-d thing, I think that I’ve pretty much done as much as I can for this box. I don’t want to overclock it any further for risk of breaking my processor down too quickly, so I'm pretty much out of ideas for performance enhancement.

[Natural ChemE]

The i7-3930k and the i7-3960x both suffer from an error in the first stepping, C1, which caused them to have their VT-d features disabled.

It’s still unresolved as to exactly what’s wrong, and I rather suspect that Intel doesn’t have all of its ducks in a row on this one. This said, it’ll likely remain a mystery as I’ve finally gotten a C2 replacement approved, and the issue’s now become moot.

I know it’s geeky but part of me’s curious to see how well the C2’s overclock. The C1 got to 5GHz pretty easily with a simple self-contained water cooler on medium speed, so I’m tempted to blow an hour trying to seriously overclock the C2 to see what happens.