Atomos Ninja-2 - Part 3 - Field Test

December 20, 2013  •  2 Comments

Please be sure to catch parts one and two of this three part post...

 

For this test I'm going to be using a few set ups to get some test results. First I'll be using both the Atomos Ninja-2 and the SmallHD DP4-EVF with my Sony SLT-a99 on my newly rebuilt shoulder rig in some real world testing. I'll also be using it to record from my Canon XA-10 and from a GoPro Hero3 Black. In all cases I'll be recording in both devices at the same time for some A-B side by side samples.

The XA-10 is a camera I keep expecting to grow out of but always find a use for. Last year it was Canon's smallest, least expensive pro camcorder (now replaced with the XA-20). It has all the features standard to a pro video camera which aren't present on DSLR style cameras making it a perfect camera to set and forget. I honestly wasn't sure what to expect from pairing the Ninja with this camera but it's on the compatible list of cameras so it needs to be tried. The XA-10 is the most storage friendly camera I have as it has 64GB of internal memory and two SD slots. With the ability to span recordings from one slot to the other, the capacity for long recordings is unmatched...especially with lower quality compression. As a favour I filmed a local seniors association christmas theatre show using the XA10. It was the perfect opportunity to see what the Ninja could do.

The first thing I noticed was the weight. Clamped to the handle of a fluid head combined with a lightweight camera like the XA-10, the fluid head had to be locked often to keep it from falling over. It took some repositioning before I found a neutral balance. Compared to the small, light XA-10, the Ninja is a boat anchor.

The other thing I noticed was that the screen was far brighter than the built in screen on the camera...not brightness brighter but exposure brighter. Images which were properly exposed (or appeared to be) on the camera screen looked blown and washed out on the Ninja. This was both annoying and made it difficult to use the Ninja-2 without the built in monitor.

I tested zebras and peaking on the Ninja and found that with both turned on, the image becomes so cluttered and noisy combined with the above exposure issues that it's nearly impossible to see what's going on in the scene. This is especially true with scenes with small, fine detail...like the wide shot of a choir.

That said, the ease of use was phenominal. The XA-10 touch screen interface is bulky and cumbersome by comparison. As a recorder, it's simply amazing.

The audio meters on the Ninja-2 are also much more advanced and finely detailed than those built into the XA10...I liked this a lot. Switching meters off on the camera de-clutters the screen some what...a good thing if the screen is already small and cluttered.

The filming took place in a community centre gymnasium with some harshly coloured LED stage lights and little else other than the fluorescents in the ceiling...a good test in other words as I wanted deep shadows...typically where AVCHD falls flat.

****As a quick aside...I'm having issues with the USB dock for reading the hard drives. As of this writing, I'm using the hard drive in a different enclosure (modified to fit the Atomos drive casings) while I go through the trouble ticket process with Atomos...which hasn't impressed me so far****

The results of this test were a bit shocking. There's almost no difference between the footage recorded at 24p 24Mbps AVCHD and the ProResHQ recorded by the Ninja! The video looks identical and only freeze frames blown up to 400% or more show the slight differences. I didn't even bother posting a comparison video...it's just not worth it. Below are TIFF rips from the footage actual size and blown up so you can see for yourself.

This is the original frame...

XA-10XA-10

And here are the blow ups of the XA-10 and Ninja-2 side by side...the only place you can see a difference...


XA-10XA-10 Ninja-2Ninja-2

If you look closely enough, there is a tiny bit more detail in the Ninja-2 though in many of the shots that just resulted in more gain noise than actual detail. The AVCHD compression does a good job of cleaning up the noise to reduce storage requirements.

I can only conclude that in this case the AVCHD codec is optimized for the small sensor in the XA-10 and that other than for workflow benefits, there isn't much reason to use the Ninja with this camera. In my case that's a relief. I can continue to use the XA-10 as I always have without worry that I could be getting more out of it.

My test with the a99 as much more controlled. In this case I set up a simple scene in my living room and recorded using several combinations of camera and Ninja settings to see what would result. At higher ISO ambient conditions, the results were similar to that with the XA-10. I think there's just too much noise when the ISO or gain is high for the Ninja to really stretch it's legs. When I set up another scene in front of some big windows and got the ISO down to 400, then I started seeing some interesting differences. 

At 24p, as with the XA-10, the difference can barely be noticed. In the first shot of my dog Ellie sleeping, and at an ISO of 3200, I can't tell any difference between the internal 24Mbps AVCHD codec and ProResHQ. In the second shot of a couple of my cameras, there are slight differences in sharpness and detail but you have to blow up the image several times it's normal size to even begin to notice. This scene was shot at ISO 400. Both were shot with a Nikkor 180mm ED Ai-S at f/2.8

This second example is just of the cameras but at various camera and Ninja settings. I was surprised to see that there were strange interlacing problems with shots taken at 30p (60i according to the camera). These issues reduced detail in the Minolta's faux leather grip and in the lettering on the front of the lens significantly. There was also a moire introduced on the ridges of the focus ring. This was recorded by the Ninja-2 when recording at 60i but not at 30p with a 2:2 pulldown. With the pulldown the images look as sharp as they do in 24p. In this example at least the Ninja-2, set up correctly, showed a great improvement of image quality. I likely didn't notice this before because I almost never shoot 30p. The majority of my footage is 24p, with 60p shot for overcranking.

On that note, it appears to be possible to record 60p in the camera and 30p in the Ninja-2 at the same time without issue. The only problem being selecting an appropriate shutter speed.

If you work with ProRes a lot, there are definite benefits to using the Ninja. With enough footage recorded, transcoding to ProRes HQ alone can take at least several hours even with a fast computer. If you're like me and do everything on a MacBook Pro, that pretty much stakes claim to your computer resources and sets your project behind from the start.

Speaking of MacBook Pro computers, the drive dock which comes with the Ninja-2 has dual USB ports. One for data and a supplimental one for extra power. The Macbook pro only has one USB on each side making plugging both in without a hub impossible.

So where does that leave me after this completely unscientific test? Well I have to say that it's an impressive piece of technology but doesn't quite live up to all the hype. As a monitor, I wasn't impressed. It does an OK job but compared to my SmallHD DP4, it really fails. The Newer models of Atomos recorders now come with the ability to adjust screen settings using a monitor calibration unit but this option isn't available on the Ninja.

As a recorder, I'm very happy with the controls and the ease of use. I like editing in ProRes so this can save me a lot of time. Even more if I edit directly off of the hard drives that the Ninja-2 records to. I have to admit that I was expecting amazing things from the ability to record straight to a very high bitrate ProRes file but the results didn't knock my socks off. I'm not sure if that's because the latest AVCHD codecs are that good or because the HDMI output is still only 8 bit 4:2:2 compared to the 4:2:0 captured in camera. I'm sure in time I'll find scenes where the Ninja puts that to good use but in the examples I've tested with, they're nearly identical. It may come into play when images require a lot of colour grading and post processing.

In the writing of this article I found a good one about recorders at B&H Photo which explains a lot of these funny terms and technologies. If you're curious about reading a thing or two about it, check it out here.

Because of what the unit can do, and despite what it doesn't to well enough for me, it's still been a good purchase and I'll be using it on a regular basis with my video work. Because of my disappointment with the monitor quality, I might have been better off with the Blackmagic Hyperdeck Shuttle but I'm not a fan of designs with built in batteries which have to be recharged...seems like a recipe for a bad day in the field. The bottom line is that this is a piece of hardware which compliments the rest of the system and doesn't have to take the place of the internal codec.

 


Comments

Dave Tameling
Hi Richard,

The Loupe for the SmallHD monitor wouldn't fit on the Ninja. The Ninja is physically larger than the SmallHD for starters. Also, the ports on the Ninja-2 are on the left hand side instead of the bottom on the SmallHD. The Loupe wraps around the sides of the monitor and uses the threaded mounts on the side to clip to. The Ninja-2 doesn't have these threaded mounts in the sides so there wouldn't be anywhere to clip it to.

That said, it might be possible to adapt a Zacuto Z-Finder style loupe but I suspect the screen is too large for this to work. There might be some 3rd party loupes available on eBay or Amazon that might work.

I find that I don't use the Loupe that much any more. It's very bulky and heavy compared to the monitor itself. It's a chore to pack it so I use the folding hood much more often these days. In fact the only time I use the loupe any more is on my home-made Zacuto Recoil-style shoulder rig.
Ting Tian(non-registered)
Nice review, Dave! I am kind of waiting for this part3 since I had some similar experience like yours with the Ninja2.
1. There is barely any IQ improvement for regular shooting scene. I have tried Nikon D800, Canon 5D mk3, while mark tried D7100, however the IQ improvement is hard to notice. It is kind of surprising that going from 24Mbps-37Mbps H.264 to 220Mbps ProResHQ 4:2:2 didn't make too much difference. Maybe chromakey work is the moment that 220Mbps prores really shines.
2. The Ninja is below average as an monitor. Just like you said, when having peaking and zebra both on, the screen is just a mess that you cannot really do any useful work.
3. I also notice the brighter image on the Ninja with different cameras. The image on the camera was having perfect histogram/waveform, however the one on Ninja is simply overexposed. I have checked both the zebra and the actual recorded file, and confirmed that the highlights are indeed gone on the Ninja clip, just similar to your experience. This has something to do with color range from the camera output. I have more detail in my point6 below.
4. As an recorder, it is easy to use with awesome features. Two NP-F slots, awesome; SSD/HDD compability, awesome; audio features, very good; UI, simple and painless.
5. Working with different types of cameras is, interesting and annoying. There is just SO many issues occuring with different set up. Say, for a Nikon D800, you will need to remove the memory card, go in video live view(camera signal will start to flicker) and then switch to photo live view and then switch back to video live view(now the flickering is gone), set HDMI output to limited range(RGB16-235), 100%size, auto resolution, information overlay off, live view time out no limit in order to record 1080p. any single mess up there will ruin your shot to either 720p or simply washed out high lights and crushed blacks. However, when working with a D600, the switching live view mode method is not necessary as there is no flickering at all. you can also record 1080p into D600 when using a Ninja2 at 1080p simultaneously. It still needs to be set like the D800 for HDMI settings though. Then, when using a Canon 5D Mark III, another issue presents: 5D Mark III does not send audio signal over HDMI(the nikons do, and i am not sure about your sony and canon xa10). Considering the form factor of the Ninja and 5D, it is just hard and ridiculous to plug a microphone or a sound device into the system: you don't want to plug it into the 5D since it doesn't send audio; you don't want to mount a mic on top of ninja neither as the whole setup will be more than a foot tall. A rig will be necessary. When using Ninja with Sony A7, well, have fun with that stupid micro HDMI port.
6. After I noticed the fact that I probably couldn't gain sharpness by recording ProResHQ with Ninja2, I was thinking at least I could still gain some dynamic range with it. Then, I found another fact that the Ninja is an RGB16-235 recorder. This brings in a huge problem since most DSLR/consumer video cams dont have enough HDMI output option to be set to fit the Ninja. For example the D800 can get decent dynamic range when setting the HDMI output range to RGB16-235 in order to fit Ninja, while 5D Mk3 doesn't have this option. I am thinking your XA10 might have the same problem.

Afterall, I was not impressed by the Ninja2. Probably I am just expecting too much (spending $750 and get the performance of a $10000 camera, lol), or the ninja is best suit for stronger sensor like the Canon C100, or for chromakey work. That being said, I still like to work with the Ninja unit since the monitor functionalities and the ability to record "potentially" higher quality video without time limit are very welcomed for DSLR video shooters. I will do more tests with Canon C100 and C300 when I have time to see if there is any new land to be found.


Cheers,
Ting
No comments posted.
Loading...

Archive
January February March April May June (5) July (1) August September October (2) November (3) December (1)
January February March April May June July August September October (1) November December (3)
January (2) February (1) March (2) April May (1) June July August September (1) October November (1) December
January (1) February March April May June July August September October November December
January February March April May June July August September October November December
January February March April May June July August September October November December