
And the answer is yes in all cases.

Now the clear cut case is the left one, which is the one we already discussed.

But we have data on a circle that could still be a main axis and

a secondary axis.

And PCA will actually give you a deterministic result, typically the

first X in this direction, second this direction.

The third one is surprising.

When we, remember regression,

it's impossible to build a regression that goes vertically because you

can't really divide this data set here as a function y equals f of x.

But regression treats the variables very asymmetrically.

One is the input, one is the output.

In PCA, all we get is vectors.

So I can easily imagine a coordinate system where the x axis falls vertically,

and the y axis goes to the left, and that's the answer for PCA in this case.