Sunday, July 29, 2007

I reckon most camera testing gurus are wrong!

Recently there have been a few people on dyxum stick their heads above the parapet and have a go at producing comparisons of cameras, lenses, flashes etc. Many of these testers have been roundly abused and their efforts trashed. The common theme from self anointed test experts has been that the people conducting these tests had wasted their time by trying to use their cameras as they normally would instead of in a precisely controlled and identical way.

I have been uneasy about the way I have seen people testing and reviewing cameras for some time but there is nothing on tv right now so I figured I’d spend a few minutes putting my thoughts on paper (well on disk to be more accurate).

I am relatively new to photography but I have been involved in testing complex systems in one form or another for most of my career. Over the years I have been involved in the testing of cars, trucks, plant, aircraft and surveillance and control systems and my experience tells me that the accepted wisdom of how to test cameras is completely wrong.

Over and over I read people dictating how a test is to be conducted – use exactly the same speeds, exactly the same ISO values, exactly the same apertures, develop using the same software, same sharpness settings etc. Sorry but that simply isn’t a meaningful test.

Imagine you were testing cars over a standing quarter mile, would you specify that both cars had to be launched at precisely the same engine speeds, that you had to change gears at the same revs, etc? Of course not. You would conduct a few test runs to determine the ideal launch and gear shift points for each car and then run each car in their optimum configuration. Then after each car had given up the best times they could achieve you would truly know which was the fastest. That is a meaningful test.

If I was testing a radar, the characteristics of the target, its position, velocity vector, RCS would all be identical but I would configure each radar in the manner that best suited it. If one radar performed best at higher frequencies and other a lower frequencies and I would use each differently were I to purchase them what would be the point of forcing each to operate at an artificial common frequency. All I would be ensuring would be that my testing was a waste of time.

Cameras are precisely the same. When testing cameras we should design the tests to ensure each camera produces the best image the camera can in a given situation. All that needs to be kept constant is “the shot”, ie the scene and the lighting. Just about everything else should be up for grabs. One camera might have great inherent noise characteristics meaning you can ramp up the ISO and make a shot relatively easily. Another might not have such great noise characteristics but you might be able to get around this by “over” exposing and then winding back the exposure in processing. That one camera might have required a little more work in a given situation is worthy of comment surely, but that is all, ultimately any meaningful test must be designed to ensure each competitor was configured and used to yield the best possible results. To do otherwise is to pretend that there is only one way of taking a shot, only one way to develop it, only one way to present it.

We know this is not true, so why would we persist in running tests that, by definition, have no relevance to how we would actually use our cameras?

I think we need to stop thinking that camera tests need to follow the same old tired formula of here is a scene and here are comparisons of iso x, y, and z. That test really does not help me at all. It leaves so many questions unanswered. Instead a good test should say here is a scene, this is the best shot we could get from camera x and this is how we got it, this is the best we could get from camera y and this is how we got it. This is a real world test with real world application.

What do others think?

I know a lot of people on dyxum place great stock in the so called “scientific” method of test, I’m really interested in why would you test that way? Do you really restrict your use of your equipment to the highly restrictive regimes you advocate for your testing?

Tuesday, July 17, 2007

Tamron Workshop 13, 14 July 07

On the weekend of 13/14 July 07 I attended a photography workshop sponsored by Tamron and run by Roy Toft, a professional nature photographer. This was my first ever photography lesson/workshop so Roy isn’t up against a great deal of competition when I say it was the best photography workshop I’ve ever been to.
But more seriously it was really good and I’d highly recommend the to anyone. I learnt a lot, but more than anything found myself energised by being around someone who so obviously loves what they do and after many years working at the highest levels of this craft is still learning and trying new things.
So what happened. Well the workshop began on a Friday night at Robi’s camera center in Tacoma. This was a pretty basic slide and talk show with Roy talking briefly about what he does but then launching into a selection of photos he had taken on a recent trip to Costa Rica. In talking about the photo’s Roy was completely candid about how they were taken and went into considerable detail concerning the problems each shot posed and how he set the shot up to overcome them.
Many (possibly all?) of Roys trips appear to double up as workshops where he takes amateurs out into the field with him and teaches them to shoot birds, bears and monkeys with real birds, bears and monkeys. I think it is this experience of teaching amateurs like me while also trying to do a professional shoot that gives Roy a special insight into the mistakes amateurs make. I know that just about everytime he said “One thing I see a lot of amateurs doing ……” I found myself nodding. The next day at the shoot I tried to follow his tips as much as I could and I found that I was a) enjoying my photography more, and b) getting some better photos (not all better but I’ll talk about that later).
So, I’ve got about ten pages of tips but I haven’t got my journal with me but here are a few of the things Roy said that really struck a chord with me:

1) Don’t feel you are cheating if you are using a long lens for nature photography. For animals in trees a long lens is actually preferable as it will lessen the angle between you and the animal and cut down on the number of butt shots you bring home.

2) When shooting animals on the ground however get as low as you can. Sitting over a lion and shooting down turns it into a cat, shooting down on a wolf turns it into a dog. While we don’t all get to shoot lions and wolfs the principle applied generally. Shooting down on animals reduces them, shooting at their level or up empowers them (and by extension your pictures).

3) When shooting wild animals don’t rely upon stumbling across them as you are both moving. Do your research, figure out where they are likely to be, get there early, sit still and wait. Consider using a blind. This will avoid a whole memory card of startled looks and butt shots as the animal is surprised and then runs. An animal going about its business naturally is a much better subject than one terrified into stillness.

4) White sux as a lens colour when trekking thru the scrub.

5) Shooting snakes with a 14mm lens is really dangerous.

6) The success of a day is measured in the quality of your best shots, not the quantity of animals you’ve bagged. If you get a good animal be prepared to really work the situation with different angles, different lighting. Be prepared to spend a 100 shots on snakes head to get the one perfect shot with his tongue out (for example). Don’t see an animal, get a shot away and think “bagged that one – moving on.”

7) Be prepared to play around a little. Great images aren’t just technically great, in fact sometimes the technical side is a little off, but they must create a great mood, convey a feeling, an ideal. We should be trying to capture the character of the animal.

8) While subject is the first consideration, background comes just behind. The difference between a good and great photo is often just the background.

9) Just because you can fill the frame with a monkeys face doesn’t mean you should. Be prepared to step back a little and try and put the animal in context.

10) Don’t feel you have to capture the whole animal. Figure out what you want the photo to say and show then work on what matters.

In addition to these general tips Roy passed on a wealth of more technical information which I’ve written down but haven’t got with me such as ideal shutter speeds for creating panning shots with flying birds, how to plan for the use of flash for freezing motion, etc.

Anyway, the Saturday was mostly spent at a wildlife reserve in Olympia (Nisqually Wildlife Refuge). There we tried to find animals to try out our new techniques. Unfortunately wildlife was a little thin on the ground however everything worked out when the group found a trail teeming with thousands of baby frogs and the odd salamander. In a flash the workshop turned into a very helpful lesson in how to do Macro in the field.

After that we went into the Olympia Farmers market for some food and more general photography practice then back to the store for general chat, handing out raffle surprises etc.

Something I have left out was that as the event was sponsored by Tamron a rep came along with 5 great trunks full of goodies for us to play with. In essence he had several copies of every just about every Tamron lens for every mount we could simply sign out and play with in the field. Unfortunately I came away from this needing another lens – bugger.

All in all a great day and a bit. On the strength of this day I would strongly recommend your check out Roy Toft’s website and I know I will now be trying to convince my wife we need a weeks holiday in Alaska with the bears in 2008.

Monday, July 16, 2007

What I've been up to Jul 07

Ive not posted for a few days, I've been too busy actually doing things to get time to write about doing things. What have I been doing?

Well first, I was not careful enough a few months ago and moved my photos around without maintaining the connections between the files and their photoshop locations as registered in my catalog. Do you know how long it takes to reconnect 18,000 photos? About two weeks, working 2 or 3 hours a night 4 to 5 nights a week. But thats done now.

Next, I've just attended a photography workshop and getting myself ready for that and then actually doing it has consumed much of my time over the past days.

So all in all I havent had time to really think too much.

I'll post a review of the workshop soon.

Thursday, July 12, 2007

Sony and the DSLR Lens marketplace

We are (apparently) on the cusp of the launch of a new range of lenses from Sony. This, the delays in Sony's release of new cameras and their stated aim to become one of the big players in the DSLR game has made me wonder how the current range of Sony lenses stacks up against the big two, Canon and Nikon.

PS:

Red cells indicate sony price over their rival (number percentage over).

Green cells indicate Sony price under their rival (number percentage under).

Note there are a lot more reds than green.

DSLR Lens Marketplace

Wednesday, July 11, 2007

Confessions of a Digital Photographer - I hated film (and still do).

Ok, some background. Cars were my first love. As I travelled on public transport going to and from work or as I sat in meetings that were going nowhere I would doodle lists of my top five cars. Top five Italians, top five Germans, five best years in American automotive history, best five cars I could ever expect to own, that sort of thing. One thing you would have noticed on my list (I will put them on my website one day) was that few, if any, of my ‘top five cars’ was new. In fact, the very newest car I would buy if I won the lottery is probably a mid 80’s Lancia Intergrale.

There is just something about old cars that I love. They are more tactile, less anodyne (in all its meanings).

But despite all this romantic longing for the automotive past, I never, ever pretend my old cars were as good as the newer ones. As an engineer there were things I simply could not ignore. The simple facts are that new cars are designed to far higher standards, built from far better materials, and manufactured to far closer tolerances and to much more exacting quality standards.

I have always loved my old cars but respect the ability of the new.

Which bring me to photography and film.

I grew up with lots of cameras in the house. When I was a teenager my parents bought me a camera hoping I would take it up as a hobby. I think I shot half a dozen rolls of film before I decided that photography blows. What do you mean it will take days to find out if the photo worked? What do you mean the light has changed and I have to change film half way thru a roll? Why are my photos either yellow or blue and why cant I change them. Here is the image I want to capture – sorry it went while I was phutzing around with a dozen technical issues. Will I take two shots just in case, nah I have to save some film – damn it didn’t work, I should have taken a second shot.

From 17 to 37, twenty years, I reckon I have maybe 100 photos, that’s it. (I have managed to fill in a far few of these gaps by going through my parents albums fortunately).

Then at 37 (three years ago) I discovered digital photography and I have taken perhaps 8000 images. I love it, everything film did to turn me away from photography is gone. I can check my shots worked, I can improve shots on the fly, I never need to worry about taking more photo’s, I only shoot raw so white balance isn’t an issue, I can print quality color photos at home, I can have prints in any shape I want, I can add text, drawings, create collages, story boards, slideshows.

And the quality is whatever I want it to be. I can control color, detail, sharpness, tone and texture. All this and I am using essentially an entry level camera, a pretty standard home computer, the cheapest software I can find and a very basic inkjet printer. I will be upgrading my camera, printer and software and I cant imagine the results I’ll get then.

Just about every review I have read of cameras as a tool has stressed one point, that the less the operation of a camera intrudes into the relationship between the photographer and the subject the better. And by this measure digital has film skun, spitted and served with gravy.

But the best handling camera (or imagery system if you include computer, printer, scanner, software, tablet etc etc) is next to useless if its photos aren’t up to snuff. Do digital photo’s match the quality of film. This is tricky as we are now in the realm of aesthetics and this is difficult to judge. There is also the issue of whether I will ‘allow’ the digital manipulation of an image captured on film (is this a film image or a digital one – I think its digital?). I will put it this way, I read that people think digital images render skins that look plastic – I simply don’t see it no matter how hard I look. People say that digital pictures may look sharp but actually lack detail – again I have looked and looked for this and I just don’t see it. I just don’t see the differences, everything I have seen shows me one thing that with a digital camera and half decent software I can produce an image that match of anything film can achieve.

If you really disagree remember that virtually every photo you see today in print is a digital image. It may have been captured on film initially but buy the time we see it the photos have been through the Analogue to Digital conversion, compressed, reformatted and printed.

So then it all comes back to the performance of the imaging package and like I said before that isn’t a contest, it’s a rout with film running hard to avoid being steamrolled..

My father had a lovely old Pentax SLR (which I have inherited and is winging its way to me right now). I also have a beautiful 59 Pentax Asahi Screwmount and a petite old Canon rangefinder. I occasionally sit and play with these cameras and wonder what it would be like to shoot with them. I’ve dry shot them each about 1000 times and got close to actually taking a photo, but every time I have stopped short. Why – film. As soon as the thought of using film crosses my mind the old cameras go back up onto their shelf, the Alpha comes out of its case and I am happy again.

I love driving my Lancia, I’m trying to buy another mid 70’s Fiat and am searching for a decent 69 Buick Riviera. I know that a new car would be more reliable, have better performance, use less fuel, be safer etc etc. But I just want my old stuff.

So I understand and respect peoples desire to keep shooting film. But I felt the need put an alternative view.

The truth is film kept me away from photography, digital brought me back – sorry but I hate film.