[Intro Music]
Hey! I'm Cam, and you are watching Reality Check.
That's right, 64% of people can't tell the difference between PS4, XBox One, and PC.
At least, that's according to a study done by me.
and, uh, some of you in fact.
Here's the skinny: Over 2,700 people took part in an online test
which tasked participants to watch 3 different gameplay clips back-to-back,
and then attempt to identify which was PS4, which was Xbox One, and which was PC.
The results were then collected over the course of a few weeks.
Okay, so a few disclaimers:
Was the test perfect? No. Was it scientific? Kind of.
Should it be taken as gospel? Probably not
But does it make an interesting point about how we value graphics? Definitely.
But, before we dive into the details of the test, reveal the correct answer,
and talk about what went well and what I would maybe do differently next time,
let's take a moment to recap and rewind.
On Monday, the 16th of February,
I ran an experiment in our GameSpot studio in London.
I got 7 volunteers to play 3 different games on 3 identical setups.
Their task: to tell me which they thought was PS4, PC, and Xbox One.
All other ways of identifying each machine were controlled for.
All they had to base their decision on was how the game looked: the graphics.
To get the full lowdown of the test and how people got on,
watch that episode of Reality Check.
The problem, though, was that the test pool size of 7 people
was WAY too small to draw any meaningful conclusions.
So, I devised a more simple one that would allow anyone, anywhere to take part.
So, how did I do it? Well, I added 3 video clips at the end of that previous episode,
all consisting of Far Cry 4 gameplay.
One clip was captured on PS4, one on Xbox One, and one on our gaming PC.
I asked those watching to vote on which they thought was which in an online poll
Simple stuff.
Now obviously, web compression was always gonna be a big problem.
As a standard for the show I uploaded the video to GameSpot and Youtube
at 1080p resolution and 60 fps,
and both were compressed using the H.264 codec.
Now this is fine for general viewing, but there is always a degree of
quality loss and artifacting, so it was far from ideal for this test.
So, in an attempt to combat this, I also uploaded all three clips
at 1080p-60, packaged up as one movie, uncompressed, to Google Drive,
for anyone to download and watch for themselves.
Okay, time to reveal the correct answer: Clip #1 was PS4.
#2 was PC, and the final clip was XBox One
This was the single most popular answer, with 982 votes, or 36%.
The other 64% of voters, though, did get it wrong, either getting 1 or 0 correct.
Interestingly though, if you shift the focus of the test to see how people
performed at picking out the PC
from the other 2 consoles, then the results actually get a lot better,
with 58% of people correctly identifying the non-console.
That was made up of the 36% who got it correct, plus 22% of people
who just got the consoles the wrong way around.
As for the rest of the results, here they are:
391 people, or 14%, voted PS4/Xbox One/PC,
311, or 11%, chose PC/Xbox One/PS4,
291, or 11% again, thought it went Xbox One/PS4/PC
And finally, 104, or 5% of voters, chose PC/PS4/and then another PC.
The eagle eye amongst you will notice that that last one was wrong.
The final option should have been Xbox One, not a second PC.
So how did that happen? Well,
that's what we call a derp. (Otherwise known as human error.)
It was very late and I was very sleepy when I made that poll, simple as that.
Whoops! But it doesn't really matter, though,
because it was an incorrect answer either way.
So, that was the experiment and the results,
but like I said, it wasn't perfect.
So it's time to look at some of the comments you guys had
but also think about how I might do this differently
if I was gonna do the test again.
One big issue a lot of people commented on was my choice of clips in the experiment.
Now I used 3 clips from the same early stage in Far Cry 4.
The areas, though, weren't identical, but this was a deliberate choice.
You see, I wasn't attempting to emulate the artificial setup
of a graphics comparison, rather,
I was trying to create real-world gaming conditions, if that makes sense.
Sure, identical clips would have made it slightly easier,
as you could have compared identical textures like for like.
But the way I see it is that texture quality, dynamic lighting, and all other
graphical settings are all consistent in a whole game area.
And, more to the point, if graphical differences between platforms really
weren't the fuss that a lot of people tend to make over them,
then we should be able to tell under the conditions I set.
That was my logic, anyway, but I'm willing to be talked around on that one
if I were to repeat this test. So let me know what you think.
The main issue for me, though, with these results is that
I have no way of knowing how many of those who voted did so after watching the
online version and how many actually downloaded the clips.
Downloaded votes are far more powerful
as you can eliminate all that nasty compression
and give people a much fairer test.
Next time I'd find a way so only after actually downloading the file
could you cast a vote.
Then, there's the case of the gaming PC.
The specs of the one I used are on screen now, and I think it's fair to say that
it makes for a reasonably powerful gaming PC, probably to the higher end of average
that most PC gamers have.
Plus, Far Cry 4 was running at the preset ultra settings.
I'll admit, though, that it would've perhaps been fairer
to have the most powerful PC I could get my hands on.
So next time I'd jam in a Titan X in there alongside a better CPU.
But despite these changes I would make in the glory of hindsight,
I still think these results are really interesting.
For me, this test simply underlines how graphics are not the most important factor
to consider if you're trying to decide which setup is best for you.
Ultimately, PS4, Xbox One, and a gaming PC are fairly close in visual quality.
Sure, you can line up labeled textures and spot the highest resolution one
in a graphics comparison,
but when it actually comes down to sitting on your ass and having a great time gaming
your ability to tell, and therefore the importance of those extra pixels
or quality of lighting, or whatever—
start to diminish.
Other things like online services, the ability to host servers,
digital distribution platforms, controllers, what your friends use—
these are far more important when deciding which is best for you.
It's all about our own individual preference.
But that's just what I think.
Do you guys agree or do you disagree?
Or do you have any ideas about how I maybe could have done the test differently?
Let me know in the comments down below or by hitting me up on Twitter @camfrazrob
Thanks for watching.
[Outro music]
So, it's time to think about some of the comments you guys had,
but also what I might do differently— that was squeaky!
—comments you had, but also think about how I might do this differently,
Oh there's my chair!
but also consider what I might do differently, if I was going
to do this test again.
[laughs from Cam and the cameraman] How did that happen?
If you guys agree or disagree, or if you have any other ideas about how I
[electronic alarm sound]