I was just wondering if NetRender offers support for Dual/MultiCore Systems, so One Computer can be assigned to render more than one frame at a time.


Niels.


ps.
I don't have NetRender, but this information may come in handy when getting new Systems.

I have posed this question to Hash during the Hash Bash.

It was stated that it is possible to run more than one instance of messenger and slave on a hyperthreaded/multi-core/multi-CPU machine. It was also stated that this can't be done with the standard install, some sort of special configuration is required.

I am currently awaiting instructions on how to do this.

I will be adding it to my 'How to set up a Render Farm' thread.

Coming soon. Hope to get to it next week.

Thanks in advance Will... Hash ROCKS!

Can hardly wait this feature would be sweet!

-Ethan

I see this thread has been quiet for a while. Did anything come from Hash on this?

I would think that if the system had a 2nd network card and a separate user login... this might work; it would appear to be a separate system... I think. I have a dual core but simply have had no reason to try this yet. Currently while I work in AM, the other core runs Render server and a slave. When I have a back log in rendering I might be interested in trying this.

Cheers,
Rusty

It would be great to have some one from Hash join us on this thread that was at the Hash bash talking about this being possible to do.

Its in the works, just not sure when I'll have time to finish. I thought I could do an easy hack to get it work, but it will require a bit more work to get it to actually work correcty.

Will,
This is great news to hear that you are working on it. Thanks for the update on this very important advancement of net render.

The multi-processor support for Netrender certainly would be good, particularly with Intel's coming quad-core chips.

In the mean time, the only solution I know about is to run several virtual operating systems on a PC - each with a running netrender slave. The class of virtual PC environments that allow machine code to run at full speed directly on the processor shouldn't slow the rendering down too much. Each virtual PC/OS has its own seperate IP address so to Netrender, each virtual PC is a seperate machine.

A good comparison of the free and commercial virtual PC environments is:

http://en.wikipedia.org/wiki/Comparison_of_virtual_machines

As soon as I pick up a bigger hard drive, I was going to do some testing on a 3GHz Sempron PC I have spare at the moment.

Some of the tests I was considering was a speed comparison of rendering on:

Windows XP
Windows Vista Beta 2 (Just thought I would see if A:M and Netrender actually
runs properly on the next Microsoft O/S.)
Windows NT4 (Did work with V11 last time I tried. Operating systems only
chews up 20Mbytes of RAM leaving lots of RAM for rendering.
I also have a pile of 16 licenses so it is a cheap solution).
Reactos Beta (I haven't tried this yet but there is a very good chance it will work
A free Windows-compatible O/S for render slaves would be nice.)
Wine + KDE (I did this ages ago but couldn't perform a real speed test
as my old Linux system was 400MHz/256MByes.
I doubt I would ever use this method seriously)
Virtual PC's running on XP and Linux.

I don't have any dual-core PC's yet, but if I am lucky, I will have some in a month or two.

Richard Harrowell.

Richard,
Thanks for the info. I will have my teck look at this link and see if he has anything to say on it.

Here's my progress so far.

I have Linux running on a 3GHz Sempron system running two Netrender Messemger/Slave sessions concurrently. All absolutely rock-solid and all with a total O/S license cost of $10! And no Microsoft Activation problems!

To step back a bit, there are a number of different Virtual PC packages - open source and commercial. The Open soure packages can work well, but the best commercial ones are very polished and reliable.

It just so happens that Microsoft has just recently made it's Virtual PC 2004 SP1 package free and when the 2007 version is released, it will be free also. Very nice if you want to run a virtual PC on a Windows platform.

Even better, VMWare is providing a number of its packages free also! Their main package is the VMWare workstation, but they have released a cut down package, VMWare Player free, and also a fabulous package VMWare Server for free (that's what I am using). It can run on all sort's of server platforms so I am currently using Linux. With VMServer, the console window for the Virtual PC can be on any PC on the network so it is wonderful for render PC's.

For the Windows, I am using NT4 SP6a + DirectX6 + IE6.

It is really easy to install NT4 in VMWare because they provide all the Virtual NT4 drivers (so you can run NT4 on a motherboard and graphics card that have no NT4 drivers!). We have a pile of sealed NT4 Workstation licenses we picked up on ebay for $5 each. So far NT4 runs the latest V13d renderslave as solidly as any other Windows O/S. The complete NT4 operating system uses about 250MBytes on the virtual hard drive and occupies about 25MBytes of RAM. When I am rendering the Toys project out at 720x578 (PAL DV), I am using a total of 150MBytes of RAM in the NT4 Virtual PC. The complete virtual PC environment can be zipped up into a 500MByte file and moved to any other VMWare system. Very simple!

Speed seems reasonable.

Athlon 1800+ Win2000 - 59 secs/frame
Sempron 3000 (single core) + Fedora Core 5 Linux
running ONE NT4 Virtual PC - 47 secs/frame
running TWO NT4 Virtual PC's - 94 seconds from each Virtual PC

I just have to install Windows directly the Sempron so I can determine the speed loss of VmWare but it is probably not much.

If I was doing this for real, I would choose an enterprise Linux (probably CentOS which is a free clone of the brilliant Redhat Enterprise Linux). The reason I am doing this is because I hope to pick up a number of dual core Athlon 64 4200 boxes cheaply from a closing business and I want to know how to get the most out of them.

The Reactos package is currently too rough to use, but in a year or two, it could easily be good enough for a free alternative to Windows for A:M rendering PC's. Perhaps just in time to take over when NT4 finally cannot cope.

Richard Harrowell.

Some more encouraging news.

It would obviously be great if Hash can release a Messenger that can invoke multiple render Slaves on a multi-core or multi-processor PC.

Until then, you definitely can efficiently run multiple renderslaves by using virtual pc's.

I did the tests today to see how fast a vmware virtual pc was.

The results on my Sempron 3000 (754 pin) single core processor:

Win 2000 Pro Installed directly on Sempron PC: 47 secs/frame
Linux on Sempron + VmWare Win 2000 Pro virtual PC: 51 secs / frame
Linux on Sempron + VmWare NT4 virtual PC: 51 secs / frame
Linux on Sempron + TWO VmWare virtual PCs 105 secs/frame *

(* NOTE: on a Athlon 64 X2 or Intel Duo 2 system, you would expect the two virtual PC's would each run at almost the same speed as the single virtual pc speed. The first generation Intel dual-core processors probably will not show much speed improvement due to their thermal speed limiting).

So the VmWare Virtual PC is only slower then Windows directly installed on my Sempron PC by a factor of 8%.

Since my Sempron only has a small cache and single channel RAM, I would expect processors like the latest Athlon 64 X2's and the Intel Duo 2 processors to lose even less speed. You definitely should be able to work both cores fully.

Linux is not necessary. You could also have, say, an Xp system running netrender along with a virtual PC running a second Netrender.

What these tests show is that:

1. A Linux render PC with the free VmWare Server is a solid A:M render machine.

2. Even with the current Netrender, you can run two or even four Messenger/RenderSlave
processors very efficiently. (It will not be long till Intel's quad core processors arrive).

3. NT4 currently is a very solid render option for a virtual PC. It renders at exactly
the same speed as Windows 2000 Pro. I would imaging that XP is also the same.

4. A single VmWare console on any PC on the network can control all the Virtual PC's.
All the virtual PC's are listed on the console and you can instantly switch between the
desktops.

The thing I still need to try is to see if Nt4 will handle all the current plugins (like Darktrees, EnhanceAM, etc).

Obviously, installing Windows XP or later on a virtual PC will require that each virtual PC has its own unique license and each has to be Activated by Microsoft. I haven't researched the consequences of this (you Xp license becomes permantly tied to a virtual PC rather then a real PC). Perhaps even upgrading VmWare might count as a new PC requiring a new license. Win 2000 or Nt4 are much better options now with Reactos becoming a free solution in a few years.


Richard Harrowell.

All I can say is WOW! Rick, thanks for the heads up and expirimentation. It totally works! I'm not a linux guy so I dreaded the thought of playing with it. So I downloaded and installed the free virtual PC you mentioned set up in no time with win2k. Network setup and everything was an absolute breeze. I couldn't believe it. I didn't know that you could have two seperate IP's coming off the same network card. Right this very moment my dual core is happily rendering away two frames in netrender.

So to break it down for the Windows crowd.

1. Download and install virtual PC 2004 http://www.microsoft.com/windows/virtualpc...nloads/sp1.mspx
2. Setup an OS in VPC2004 (pref. win2k) It's guided and VERY easy to do.
3. Install NetRender on your new little Virtual PC

and away you go!

Sheese... could it get any better??

Thanks alot rick!

-Ethan

Very cool.

What about Pentium hyperthreading machines?

Can you run a virtual PC and have it use all the processor?

Thanks for working on this guys.

David

It wouldn't be worth it to install two virtual PC's just because you have Hyperthreading. Hyperthreaded Intel CPU's are still single core with a single FPU( floating point unit) so when Task Manager says it is running at 50%, that is pretty much the best it can do for A:M.

The overheads in running virtual machines would eliminate any slight extra speed you would gain by running two seperate Renderslave threads.

I would only set up multiple virtual rendering PC's it on genuine multi-CPU motherboards, AMD Athlon 64 X2 based systems, or the recent Intels Duo 2 based motherboards (and especially their new Quad core processors out soon). I don't believe that even the first-generation Intel dual-core CPU's can hack it because I suspect that they cannot run both cores continuously at full speed, even with the best of cooling systems.

Richard Harrowell

Ethan,

Its fun, isn't it?

I haven't tried MS Virtual PC yet. Any chance of some benchmarks to compare the Virtual PC speed with the native PC speed? What kind of PC do you have?

One of the best things with Virtual PC's is that the whole virtual PC is just a file - to set up another virtual pc you just copy the Virtual PC appliance files and run them.

I still have to study the licensing requirements - does each Virtual PC need a new Windows license or do you just need one for the PC, no matter how many virtual PC's you are running?

As far as distributing the one Virtual PC appliance to multiple render machines, I seem to remember that legally, you can run multiple copies of Windows all using the same installation license code, as long as you possess a seperate license for each installation.

Richard

It certainly is!


I did do a few render tests, these are done on a AMD 64 4200 X2 with 2GB RAM. One thing to note is that my virtual PC only has 256 meg of virtual ram, leaving 1744 for my system. Also, the real system is also acting as the render server. This could affect performance too.

Case 1:
Used both slaves at the same time. Consecutive frames.
Core1 (10:29)
Virtual PC (11:55)

Case 2:
Slaves one at a time
Core1 (10:24)
Virtual PC (11:24)

Case 3:
Smaller Resolution Frame
Core 1 (3:54) 15 of these seconds for loading the scene
Virtual Core (4:44) 30 of these soconds for loading the scene

As you can see there does seem to be a bit of speed bleed off. But it's not to bad. I wonder what things could be improved to help this out.

-Ethan

Tha's great! Within the week, I will have my hands on exactly the same configuration (AMD 64 4200 X2 with 2GB RAM) and I will be using my Linux/VmWare approach so we can compare results. In fact my work on the Sempron system was a trial run for the AMD 4200 system.

It looks like you are loosing about 10% to 14% speed in the MS Virtual PC compared to about 8% for me using VmWare on a Linux box, but as you pointed out, you had only allocated 265M RAM whereas I had 400M Ram in my VmWare Virtual PC's. I was also using NT4 which left a total of 365MBytes free for RenderSlave to run in. You would have had half that.

I doubt that NetrenderServer will use any signifigant amount of processor time.

I also have to try the free VmPlayer sometime on a Windows PC.

Richard Harrowell

Are there any concerns that the Virtual PC's are going to run low on RAM?

I run into problem loading larger projects, and having the slaves lock up.
Granted I am making a jump, assuming the lower amount of RAM and larger projects could be a problem.

David

If a normal render slave can lock up, a slave on a virtual PC can also lock up, I guess.

Obviously with the RAM, you have to provide enough RAM for two renders, but it is easy to check the RAM usage with Task Manger or Perfmon.

If you are getting a very slow loading of the project, I suppose you could put a complete copy of the project source files on every render PC (including the Virtual ones.

By the way, I found out a bit more about the free VmPlayer.

It cannot make new virtual PC's, but it can run virtual PC's created by VMware Workstation, VMWare Server or even MS Virtual PC 2004. So if you are using MS Virtual PC, you could try loading VMPlayer and see if it runs better. VmPlayer can also be distributed free (as long as you follow the terms in the license) and so you can make complete virtual PC's that can be redistributed.

So for example, it may be possible to have a compelete redistributable Linux PC running Cinepaint (OpenEXR compatible Movie frame editor - widely used in the movie industry) or Cinlerra Compositor/Editor (I think it is also OpenEXR compatible). Since it is a virtual Pc, it can be run in Windows without a problem.

Richard Harrowell.

Although Virtual PC is a workaround, is there and confirmation that Hash is working on programming multithreading into a new version? I want to avoid using virtual PC setup just because it seems like more trouble than its worth. Don't get me wrong, it is cool, but it would take a long time o set up on multiple machines with dual core CPU's.

Right now I am up to (2) Dual core windows machines. one is a Macmini with the core duo running boot camp ( Runs great by the way ) and the other is the New Intel E6700 Core Duo 2.66. (Mega fast, I'll tell ya).

I also just rdered a New Shuttle SN27P2 box and an AMD Athlon 64 X2 5000+ processor. So basically everything in the house is dual core. All this power is just for working on AM. It seems logical that multithreading Animation Master at this point would be a huge benefit to a Lot of people. Most new systems coming out now are dual core and rendering times can obviously be improved by multithreading. I love Animation Master and I think it would benefit a Lot of users to multithread.

My 2 cents.

I spoke to the A:M programmers at the SIGGRAPH show (great guys btw) about Dual Core machines, and they indicated it was coming. I am not speaking for them, but they understand that with the plethora of multi-core machines around, this feature will really help.

David

Definitely, dual core processor support for rendering PC's would be great.

It is nice, though, to have the Virtual PC solution as well since Netrender is usually not high on the Hash priority list and I suspect that this would be a major Netrender re-design.

The good things about having Virtual PC's available is that it means that if you have, say, a compositor that can render on Linux PC's, you can have a Linux Virtual PC on your Windows render box. (For example, the free Cinelerra movie editor/compositor can render on multiple linux boxes).

One big problem with VMWare Virtual Server is that it is free for one VMWare Server on one and only one PC (unless you have explicitly obtained permission from VMWare to install it on multiple servers). I haven't checked the license for MS Virtual PC - I am still to try it.

Virtual CP package like QEMU are completely free, but they will not be as nice as VMWare and I don't think it is as fast either.

The solution is to use VM Server to create the virtual PC's and then run the free VMPlayer on the multiple render PC's - that is legal, I think.

Legal, that is, as long as you are not offering your rendering farm to any third party - that violates the liensing aggreement for the free products.

Richard Harrowell.

I have the V12 version of the net render dongle and I have not upgraded yet as I am waiting for dual core support. I just recently bought 2 new computers, BOTH with Dual core processors. One is an Intel E6700 dual core and the other is a AMD Athlon X2, 4600 AM2 processor. I am running V13f on both. Both computers have 2 gigs of ram.

Right now, in stead of running Net Render, I am just opening 2 instances of AM on each machine and starting frames 1 and 2 on one computer and frames 3 and 4 on the other computer, then Step 4 on all. Then I just click Go on each AM and this basically enables 4 rendering sessions at one time.

I also intend to add my Mac Mini in to the mix. It is running Boot camp and it is also a Dual Core 1.66 core duo chip. Rendering this way isn’t Easy per say but it is better than waiting 4 – 6 times as long. For Dual Core Chip users, rendering this way is a must. In my opinion, rendering one frame at a time on a dual core machine is a waist. Your processor should be pegged at 100%, especially if your rendering 300 framed in HD resolution like I do.

my 2 cents.

What is 300 framed in HD?

I agree with everything you said except waiting for dual-core support before you upgrade.
Upgrading helps fund the things we want. If there is not money in the coffers, it is hard to justify the programmers time. NR is a low enough priority as it is (not a gripe, just a reality).

That is my 2 cents...

David

Oops, I ment 300 frames (TGA's) in HD resolution, 1280 x 720. I agree with you about upgrading my key for netrender. I will eventually.. within a couple of months. probibly sooner. I do like that they added Mac Net render support.