Elissa Frankle is the Social Media 
Strategist and Community Manager

at United States Holocaust Memorial 
Museum here in Washington DC.

The title of her talk today is 
"Making History with the Masses:

Citizen History and Radical Trust in Museums.
So please join me in welcoming Elissa.

(Elissa) Before I start I just wanted 
to thank you, the fine people

here at the front for inviting me in.

As a Social Media Manager
I don't usually spend a lot of time

talking in front of the audiences anymore.
As I am thinking of the community

behind the computer. This is a really
treat for me to actually be able

to come out and talk with my voice about
things that are important to me, one of those

a very large citizen history
in a world of our users, and the work

we do, as museums and cultural organizations.

One of the things that is really important
in all of this is just to look at

the words that we use when we're
talking about the way we interact

with our users. So, in a sense
what we're going to talk about today

is really what is citizen history?
Not just "what is citizen history"

as a concept, but what is citizen,
what is history? And what's a museum?

Really big concepts, really interesting
things and I don't promise to have

all the answers today, because 
most of these cases, there aren't

real answers. That is the best part.

But we're going to try and get a little bit
of the why to explore some of these questions.

And see where we could get it
unlocking what would be the critical question,

of what is citizen history,
and what can it be in the future.

So Citizen History kind of came into being,
from it's early origins in Citizen Science

and Crowdsourcing. Two other ways that
other fields have looked at using their

public, to get down and dirty with their idea.
We'll look at each of these in turn,

first of all, starting with crowdsourcing.
Now, when you go to look at crowdsourcing

on the internet, one of the first sites 
you'll run into is crowdsource.com

Not surprisingly. And they promise 
500, 000 workers on demand.

And what they promise for those workers
is that your data will be dealt with -- with results.

In a speedy manner. So really using the crowd,
using the number of people you can just get

cranking away on some amount of data,
some amount of rote tasks,

to produce whatever the desired result is.
So the question here with crowdsourcing

isn't so much about big answers
and big interaction, but it's more about

a lot of people doing a lot of little things.
Museums and local organizations apply

this crowdsourcing principle in a lot of
different ways. One of the projects we're

talking about at lunch actually is
New York Public Library

What's On the Menu Project, 
and it's companion project

recently released, or about-to-be released,
the Ensemble Project.

But in this case, transcribing menus,
and the other case, in transcribing playbills.

Taking what's on the menu, what is on the playbill,
written it down into it's component parts,

just saying, what do you see here,
what is the food that you see on this menu,

and have someone transcribe that,
by some user. As a result, again, small task,

just transcription where you look at it,
what is it that you see, you write down

whatever it is that you see.
No real depth of thought

going into to it, but again, a lot of
people working on a very small task

for a long time, creating big results.
The other form of crowdsourcing

that we see quite frequently
in cultural heritage organizations

is the idea of, not necessarily putting
lots of small tasks into play,

but working more from a 
knowledge base, that the person has --

the user have some kind of knowledge
that is personal to that person,

that they then share with the Cultural
Heritage Organization.

So again, not a lot of deep thought,
deep interaction with content,

but a lot of sharing up, personally.
So rather than citizen history,

the topic of what we're going
to talk about next, we have the history

of citizens, growing on this kind of
crowdsourced environment.

So if you are going to talk about crowdsourcing
we're going to talk about all these things,

with framework in Bloom's Taxonomy,
this is an educational philosophy

framework developed by Benjamin Bloom.
They talk about the different ways that

students can engage with learning. 
Everything from just remembering,

kind of that rote level of "I see what it is, 
I think about it, I write it back down"

So the regurgitation model of looking
at that knowledge, they're understanding it,

being able to classifying things, 
up to application, they are able

to choose to interpret, to draw
some kind of conclusion.

And all the way at the top, to creation.
Starting from scratch, creating a product

all by one's self. Crowdsourcing,
we tend to think it comes down,

about this remembering, understanding,
basic level of proposition.

This is not to say there's not value in it,
but it is just, it is very much on a rote level.

I see what I have in front of me, 
I take it, I transcribed it, I translated it,

and I spit it back out in a usable format.
I have the knowledge in my head,

I have some stories that I want to share
that I've been asked to share.

And I take it out of my head, 
and I write it down, and then to you.

So crowdsourcing, microtasks,
on a macro scale.

So lots of small things, lots of people together,
sharing their personal knowledge, or basic skills,

really relying on that wisdom of the crowd.
So by having a lot of people working on

something together, eventually something
will be completed, and answers will be given.

Citizen science goes a little bit higher up, 
[inaudible]

We're going to look now at two projects
From the Citizen Science Alliance,

or the 'zooniverse' family of
citizen science projects.

Here we see Galaxy Zoo, where
the Citizen Science Alliance

and its partner organizations 
have pictures of galaxies.

And they walk through a four step process,
where they ask questions about what

the users see in these galaxies. Alright?
Are they round? Are they spiral?

What kinds of bulges do you see?
Just being able to classify what it is they're

looking at by sight. Similarly we have 
Planet Hunters, this is a, well,

from their tutorial, where they walk through
premises on how you can identify a transit.

Ways in which these levels that we see here,
dip down, when a planet transit is identified.

So we have again the small idea of looking, classifying, 
making a note, but in both these cases

we also have this very exciting thing
that is a "free text box", where someone says

"Do you see anything that is of interest,
is there anything that you want to discuss,

from what you've seen?" So more than just
seeing, repeating, replicating, we have

the ability to discuss, to take things 
to a higher level, to really reflect on

what it is that we're seeing. 
So crowdsourcing, again, down

at that lower level of Bloom's Taxonomy,
citizen science is the ability to go

a little bit higher. Thinking about applying 
the knowledge that you have,

what you gained from doing the project,
thinking about science on a larger scale.

So our basic principles of 
Citizen Science say these these are

volunteers, non-specialists,
people who are not trained in science

Governed by and under the leadership 
of people who know what they're doing in science,

and have that training, or that title
of scientist, to answer real-world questions.

Because scientists don't want people 
to just look at galaxies for their help,

though they are pretty just to look at anyway,
they want people to look at those galaxies

so they can classify them and
know more about what's going on

out there. In one article that I read 
about galaxies, they mentioned that

they first know what's successful when
they classify the amount of time,

the amount of results found by these
citizen scientists, and the number of

person hours that would have taken
for the original researcher who was going through

by hand, on his own, looking at all these
galaxies on it's own, to go through,

and make these same distinctions.
They can do about fifty thousand a week,

seventy thousand in the first two days,
so it's a lot of things that you can do.

Again, small tasks, macro scale, 
lots of people, find the answers.

So it seems to be a win-win proposition
for everybody. Professionals get data,

volunteers build skills. They learn how to
look at a galaxy, what is it that they are

looking at when they look at a galaxy.
How you identify it, the transit of a planet.

So the real skills that a scientist use to
try and answer some of their questions,

these citizen scientists actually get to use
on their own. So everybody wins, alright.

In 2006, the United States Holocaust Memorial
Museum opened an exhibition called

"Give me your children: Voices 
from the Lodz Ghetto" This was an exhibition

built around a student diarist,
child diarist, who then lived in the Lodz Ghetto,

after 1940. One of the artifacts that
was part of this exhibition was an album.

An album of 14,000 names, signed by the students
of the Lodz Ghetto, presented to

(Mordechai) Chaim Rumkowski, who was the 
administrator, on Rosh Hashanah,

the Jewish newyear, 1941.
So we have this incredible artifact,

this album full of signatures, and
we knew nothing about it.

We knew that these were students
who had signed their names.

We knew that they were about 
thirty or so different schools

who had students sign their names.
And we had another document that

gave us some remark to how old
these students were in each school.

But, the question that we asked
as we brought this album forth

was could you have
today's students, look through

our data for the things that
we would normally be used

as researchers at the museum,
and try to figure out who these students were,

as well as what happened to them.
This was really an experimental project,

the question wasn't just "What happened
to those children?" but would it actually work

to put today's students
at work, trying to figure out who these

students of [yesterday] were.
Seven years into the project

we still have this new experimental
citizen history project.

We're still very much in beta, 
we're still trying to figure out

where all the lessons are.
But we do at least have a platform.

Here I'll show you the URL for this
on the next slide.

This is the Children of the Lodz Ghetto
Memorial Research Project,

we have, at this point, about
8500 names available for research.

We have them up, transcribed 
in the database, and our student users

and volunteer users go through, 
select a name they want to research,

and then go into our databases and see
if they could figure out who the person was,

who most likely sign their name in the album.
Then figuring that out, figuring out who

their most likely candidate is, going through
outlets even further, to see if they can

figure out what happened to that person,
after the Ghetto. Were they able to

survive the war, do they perish, where,
if so. So we have, as we seen in other

crowdsourcing and citizen science projects
up here, we have a framework

where we ask you to put into our research.
What was the name that you found?

What was the date of birth if there was one?
What street addresses did you find,

associated with this person?
And we also have this all important

free textbox, where we ask, not only,
how was it that you're able to come

across who this person was, but talk
to us about the process.

What was it that made you realize 
that this was the right person,

as opposed to some other [inaudible name].
How did you know? What was it

about the document, what can you
determine about the document?

So having done again, the higher order
thinking of "What do we do, when

we look at documents?" and 
"What can we know from the document?"

and "What do we simply not know?" We've seen from the document that a lot of students like to jump to conclusions

that "Oh couldn't find anything else 
beyond stage 1, this person clearly

must have perished in [inaudible]
there's nothing else to be found."

Quite frankly the answer to that is,
well, no, the only thing that we know

we can't find the document is that,
we don't know yet.

The document just isn't there.
Doesn't tell us anything, just tell us

there are big gaps. I want to talk
about these big gaps momentarily.

They themselves are actually a big part
of citizen history museums.

So, going back to our friendly 
framework of Bloom's Taxonomy,

keeping crowdsourcing down here 
at the lower level, citizen history tries

to go even higher. Getting people not only
to analyze a text but also to analyze

their thinking, to reflect on what it is that 
they are doing. And really recognize

they are building skills. In addition,
they are still going through, helping us

researchers try and answer these
big questions in history.

So we put a lot of our trust in their hands,
put a lot of documents out there,

and then ask them to reflect
on their process, and on the process

of doing history in general.

So, knowing that much, knowing 
our framework with this project that

we have, let's return to our title
and talk about some words.

Because we present today only one
possible framework, one possible

working nature of citizen history. 
There are a lot of best practices

that we could draw from this,
we all have to go back to the words

that we use. For instance, what is
a citizen? Citizens, we usually talk

about them as citizens of nation,
citizens of a group of people,

who are members of a certain group.
And these citizen have two things.

They have rights and they have responsibilities.
Well, we museums, we're really good at

responsibilities. We're really good at saying
"Please, come in to our museum space,

But here's all the things that you can't do:
don't eat, don't drink, don't smoke,

don't take pictures, don't poke
the priceless raw files."

But, what is it that we can give our 
visitors, our users, the people who

come in our space, as far as
the rights go. We're not particular

good at saying "here's what you can do,
with our stuff." So if you are actually

set out to create a citizen project,
what we need to be able to do,

is to give people both responsibilities 
as well as rights in that space

that we create. Furthermore, 
going on to history.

History, in this case, we have to 
take within the framework

of history in a museum.
Since history is really messy.

There's a lot of different theories
on what history is, as far as I can tell.

History itself really has
no big answers, no big truth.

History, as it stands right now,
is just based on the documents.

The interpretations that we had 
at our disposal in this moment.

So that they change tomorrow,
when a new archive is open,

a new interpretation comes along,
something that makes us rethink

everything that we've ever thought
to be true, about a certain part of the field.

History takes interpretation, and history
is a constant asymptotic approach.

To the truth, without any expectations
that it will ever achieve the truth itself.

That one big knowledge about
what history is, or maybe.

Museums don't really like messy.
We like to be able to put things

up on our walls, put the text up and
leave it there for a long time.

Now whatever the interpretation is, 
that we have to take from this original data,

from our understanding of history,
we could one frame, and that's

what we put up. Hanging on the walls
and say, "Here you go visitors,

this is truth, this is what happened in this
historical period." As we are really good at

broadcaster model, we're not particularly
good at listening back.

And hearing all the questions people
might have, say look at this one interpretation,

that we have put forward, about history.
So when you're talking about the opposite

of the broadcast model, the idea that 
history is messy, there are

no answers, we want to be able to have 
citizens in our space. Really get down

to questions of trust. 
Museums often say that we are

instruments of public trust. The public
places a lot of their trust in us,

to be able to say, this is fact, this is truth.
You're coming to my museum,

to learn something, and you'd expect
that the knowledge being just

passed down to you, given to you
and you'll osmose it, from looking

at our wall text, and seeing our artifacts.
And that what you'll know.

But of course, we now know that
history is messier than that.

And simply heading down one 
interpretation, one framework,

is not sufficient. It's just one way
of looking at things.

But if museums were actually 
going to open up all these interpretations

of history, all these different frameworks
and ways of going about it,

would that then, hurt their ability
to be instruments of public trust?

By trusting the public, it then help
correct our image as organizations

that can be trusted in society.
We kind of have this Circle of Trust,

that we keep on down low,
and inside our own frameworks,

among our own staff in museums.
And in the Circle of Trust we have

often the really scary things that
we don't really want to talk about.

Like the fact that we don't know 
everything. We like to pretend that we do,

but we really don't. And there's a lot
of information or questions in our

collections where there's answers
might be, we just, maybe, haven't

gone through our collections
as deeply as we might like,

because there's a lot of them. There's 
a lot of stuff out there, there's

a lot of data. It takes a long time to
get through it. There might be answers

out there that will completely change
the way we present this information.

<i>Whispers</i> [inaudible]

And the fact of the matter is,
that as we answer these questions

we're not going to find any big truth,
any big answers, again, this constant

asymptotic approach to what the truth
might be, we're just going to find

more questions. We're just going to have
an even further path ahead of us.

But we really don't like to talk about that,
so you should know it well enough.

We place ours -- it's kind of hard to 
see here,-- but there's a big red brick wall

around this circle of trust, because 
we don't like to talk about it, or to share

it with the public. But what if we do?
What if we actually accept that there are

people out there, who wanted to know
that we have questions. Who want

to know what's still out there to be seen
and to be discovered, who realize that

museums maybe don't really know everything.
And they're really curious about what's

sitting inside that Circle of Trust.
What haven't we explored yet.

So, what if the museum said,
"well yeah, there's a lot of really messy

stuff in there, things that we haven't 
explore, a lot of questions, that we still

have to go through? And then we 
actually take the curiosity of our visitors

into play, they actually say "Well yeah,
we've got questions too.

And we've been trying to ask them, 
you just haven't been listening to us."

Well we have to warn them first,
it's kind of messy in there, it's really

kind of scary. And as we help them to enter
the Circle of Trust where we keep

all of our questions and our data,
and our unknown unknowns,

those questions that lead to further questions.
There's places where we have no data,

there's things that we're really curious
about, and we wish that this one more archive

would open up, that we'd be able to get to their stuff.
That might have some of those answers.

There's places where there are gaps
in the record.

We wouldn't just sign our visitors
into there, completely unequipped.

We'd give them a tool kit,
we'd give them some binoculars,

so they'd be able to look closer at things.
We'd give them a wrench,

that they can actually go through
and tweak the data, see what

they are playing with, messing around,
in the stuff that we have,

as well as a hardhat, because, well,
who knows what will fall out

when we actually shake the history
and what's in there.

So this toolkit are the things that allow
citizens, our visitors, our volunteers, our users,

to enter this space, this Circle of Trust,
the things that we're really curious about.

To enter into our questions and into
our data. Working in partnership with us.

To answer these questions.

Some of these when we look at citizen
history, are the questions historians have

for themselves. The ways that historians
do history, history as a process.

So how does historians look at a source?
What's available to us in the source

and what's the context for it.
What questions are we trying

to answer by looking at the source.
What's new? What might we be unlocking

with this source, what are we looking at
that might not have been considered before?

What's in your interpretation, a new 
piece of data, it's pointing us

in a new place. In the case of the
Children of the Lodz Ghetto project,

we've been able to identify a couple of
these pointers. Then our citizens

as they go through try to identify these
children, have an easier time in

going through our stuff, because we know that 
naming conventions in 1920s and 1930s

were a little different than you might expect
here in the States, because your

average student would have a Polish name,
and an Yiddish name, and probably

an nickname, maybe even a middle
name. All of which could be used in

any number of documents. So then
you'll be able to accept there are

a lot of names for the same person, 
helps people to be able to read sources

and jump to fewer conclusions.
Be able to be more open,

to different interpretations and
different names that maybe out there.

In addition, we're working with a mostly
American audience. So being able to tell

our users that in these documents
you'll going to see the day first,

and then the month, helps them better
to unlock what it is they're seeing.

And instead of putting their American lens
onto it, have a better understanding

of what it is they are actually seeing.
So, thus hardhatted, and wrenched,

and binoculared, we send our users
into the Circle of Trust, and while

we're at it we might as well jump into
that Circle of Trust.

We might as well bring the museum
into that Circle of Trust, accept that

we have questions and more data
and unknown unknowns.

And we're all in this together.
And a funny thing happens.

Because rather than being our usual
broadcaster model museums

just going out and say, "Here's truth,
take it in." We actually have conversation.

We have users talking to the museum
and the museum talking back.

We have users talking to one another,
helping each other to grow through

their research, and as these questions
and conversations iterate back and forth,

back and forth, we actually have 
more growth than we would've had

when we're just a museum talking
to itself. Or just users speaking to one another.

Because the museum still have
a really important role to play.

We are the scaffolders. In addition
to giving people our questions,

our honest research, our data, 
we're the ones who can help our users

to go from just coming in out of curiosity
to actually going out with a skill set.

Things they can use and apply in their
own lives beyond just the Circle of Trust.

So what do we get out of this?
When we open up our users

and the museum itself to accepting
we have questions, data, and unknown unknowns,

the museum gets connections. Connections
among their [inaudible], again,

kind of a crowdsourcing model of lots of people
looking at our stuff, at the same time,

drawing, from the wisdom of the crowd,
some of these answers.

We do get some of these answers to
some of these questions that we have

and we get more questions, of course.
Everytime we try to answer a question

we just end up with more questions 
and more directions that we could

take our research in.
And perhaps these are questions

we haven't considered before. Because
we've got people coming in with fresh eyes.

Looking at our stuff in ways we might not 
have considered before. And thus

where we would already have more questions,
we have more and more questions.

It's great! So what our users get out of it?
Now that the museum's gotten all this

good stuff from the people who work 
in their data. Well, the user discover.

What we know, truth about history.
That there are no civil answers, that

history is messy. In a lot of cases they
also get a very personal connection

to the history. We've discovered that
from our users at least.

We have students working on research
about students, they get very personally

invested in looking at these individuals,
their lives, their families, and what happened

to them. So having a personal connection
to this one aspect of history often helps them

being a greater personal connection
to the rest of history as well.

And frankly, we don't ask them
to give back their hardhats, their wrenches,

their binoculars when they leave.
We let them keep it.

So they take all of these great skills
they have developed, within

the Circle of Trust, within the museum's
setting, and take them out into the world.

Because really what's at stake here
isn't just citizens being citizens of our sphere

having rights and responsibilities
where we are, but it's about their

citizenship. One of the great things about
the study of history, the process

that we go through as we look at 
history, is that a lot of the skills

that we use looking at the document,
making an argument, talking to one another,

are also skills for the public sphere.
And on the internet today, it's kind of

a murky monkey place, where there's 
a lot of debate and dialogue going on,

without a lot of people talking to 
or listening to one another.

So what if we're actually be able to
go into this digital area where our

citizen lives, dig people out, you know,
have this skill set of being able to

look critically at a source, think critically
about what they're hearing, and being able

to form and close your argument,
having send them back out to the murk

of the internet, and see what happens.
See if we could actually improve

civil discourse, by having this new 
generation not of trained historians

but of people trying to think historically.
Take their skill set back out into the world.

So let's go back to our words. 
Citizen history and radical trust in museums.

What does this mean for best practices
for citizen history? Well, museums,

we have to remember that we're more
than just our four walls. That we are also

the additional space for the people
who come in to our walls.

They need to be able to think beyond
just what we want to present.

In this straight, closed box. They are
to think about the larger conversations

going on around us, in the world at large.
History is living, breathing, growing --

something that is constantly evolves.
In an early version of this talk

I didn't have history made history,
history is shared. History is noise,

and that was more active than just 
the static noun, of history.

Because history should never be static.
So the knowledge that history is constantly

growing and evolving and changing,
and the history of today would change

versus the history of tomorrow.
Also means that when we have our projects

going on we need to be able to take
whatever it is that we're learning,

and reiterated back into the project.
To be able to have the assumptions

that we make for our citizen users 
grow and change, something learn

more and more from.

Citizens have rights and responsibilities
in your online space, you've gotta be able to

let them in. Because it's not just enough
to say "Come in and look at our stuff

precisely the way that we want you to."
We have to be able to give them the right

to go into our data, muff around and see
what they are curious about within that

framework, and send us their questions 
for whatever it is that they've uncovered.

Trust is hugely public, as we just talked
about, it's really the Circle of Trust,

the idea of the public trust, and the fact
that opening our trust to the public

doesn't break down our trust.
It's as if it's becoming a partnership,

the way that we can all grow from working 
together. So we have to be able to

welcome our community into our questions,
and be able to, willing, to take our authority

out just enough to be able to say,
"Alright, what answers do you have?

What questions do you have for us,
what can you do to bring in to our sphere,

to help us all grow together."
And frankly the all important word, and.

It's really bridging here, not just citizen history,
and radical trust of museums, or just

citizens, and museums. It's really about 
partnership and dialogue.

Whenever we look at this, it's not just about
two things working and cross purposes,

it's people who think they'll be working
together. In a partnership.

So not only it's this about our citizens,
it's also about what the museum must do

for these guys, so we have to be able to
scaffold the skills we want to build,

we have to be able to engage our users.
This community takes a lot of caring

and feeding, a lot of time. To be able to
make sure people are getting the skills,

building the skills, learning the things
that we'll hope they'd take away from this.

And be able to say "We may not have the
historical authority in this space,

we have the understanding. How you go 
about, thinking historically, let's help you

grow, let's all move along this continuum 
together. So, finally, instead of best practices

I think about from these different ideas about
citizens, history, and museums, and you'll

be able to start with a question that
begs an answer. Something that is actually

a legit question in history. Something
not just to give people busy work

and say "Go." This is gotta be something
that museums are actually curious about.

Furthermore, we'll have to be able to
welcome these fresh eyes into our stuff.

We don't need everyone to be trained 
historians right off the bat, but that

there's actually value in having people
not necessarily worked with this data,

with this period of history, or with these
historical skills before, coming in

and looking at our stuff. We need 
to be able to iterate and dialogue.

Again, keeping in mind that this is
never static, this should never stay

in one place for very long, that our 
projects need to constantly be

evaluated and reevaluated, taking
knowledge that we've learned,

putting it back into the project,
and remembering it's always about

the dialogue between the museum
and it's users. Between users and users.

The conversation that goes on in that space
is just as important what we find out

from it. We need to make sure that
there is that space, for debate and discussion.

We've got some place for these people to go,
to be able to talk comfortably

to one another. We have to be able to
create opportunities for growth,

as people find that they are getting 
more and more into these skills,

learning more and more about
what they are doing. We need to

make sure that there's some place 
for them to go, beyond just the basic

level of citizen history. In the Lodz Project,
for instance, we have a level called

expert reviewer, when users have gotten
really good at doing the basic research

that we ask them to do, we can then elevate
them to the expert reviewer, and then

as a result, they are then asked
to go through and review the research

that their colleagues, their peers have done.
We elevate peers to a higher level,

they then go talk to their peers
as greater authority figures,

thus giving them a little bit more
empowerment and also give them

their -- here's an opportunity to realize
that there's opportunity for growth.

(Student) And what's after that?
(Elissa) What's after that?

That's a great question. Once you worked
at the expert reviewer a little bit more,

I'm going to go find out.
That's part of our next considerations

a little bit more. And frankly this 
community need a lot of caring

and feeding. You gotta make sure 
you've got a community of manager

that is really, willing to be boots
on the ground, constantly working

with your people, with your users,
with your citizens. And being there

to answer their questions, to help them
get through the murk of the unknown

unknowns, you know, there's still
value in there. Citizen history has

truly been one of the great lapse 
of my professional life, and the more

that I talk to users, learn from users,
understanding this that we do,

the more I like our users, the more that 
I love having them in our space,

to be able to learn from them.
And because you today are my citizens here,

love to hear if you have any questions?

<i>Clapping</i>

(Host) Sure I got lots. Thank you for giving us
an idea of what you do, and beta now

you are at it for seven years. You talked about
museums as if there is this, sort of,

global museum for certain different museums,
but even within the Holocaust museum,

could you talk about how, what kind of
sponsors is supporting, and sponsorship

you've gotten from curators, staff, directors,
boards of trustees, sponsors, members, doners?

(Elissa) Well this is little bit of where that
radical part comes in, those words in the title

that we didn't talk about. I kinda dispense
the word radical pretty early on

in the revision process because this is
really what it seems to be all about.

(Host) It's causing radical anywhere.
(Elissa) Right, but within the framework

of the Holocaust museum it kind of is.
We're still very much nervous about

having anybody who wasn't us working
on our data, one of the reason why it's been

in beta for seven years, because we're
worried about saying "The museum

is doing this project where we're putting
our data our there, come be part of us,

and look at whatever you want."
Because some elements in the museum

are worried that they are going to ask
for, where data was gathered.

Basically we are not necessarily ready
to have, out there there aren't very -- yea.

We often got a lot of support from
the educational community.

Because they might have been kind of 
on [inaudible] again, for seven years.

Then when do the people find out
about it, it's been a lot of fun

in the last two and half years after
we've mentioned it, the more people seemed

to like it. We really appreciate the fact that
we give people empowerment within our space.

We see a lot of opportunities for it, within
educational, formal educational setting.

As far as doners go we haven't really pushed
to it that much. And now that I set

the marketing department, there's definitely
more opportunities for us to do that.

About a year ago we went through
a completely revamp of the site,

the screenshots that I showed earlier
are from the new version.

And the plan was always going to be
that once we got it to that point,

we're going to release it out of beta,
and good lines from marketing

will do something to push around it
and we will get lots and lots of users,

that would be wonderful, and we would
just go from there.

Part of the reasons is an accident 
on timing. This is our 20th anniversary year

and probably 90% of my time has been
spent on working on the 20th, working

our outreach around that.
My other kind of customer have also been

spawned for that. So maybe if we done this
the year before, we'd actually be able to

run through the marketing cycle 
and see what happens.

(Host) Here's some few more numbers --
(Elissa) Sure

(Host) How many people have contributed
to that large project?

(Elissa) So we have about 1500 people
working on the project, in some capacity

or another.
(Host) Is that number increasing or decreasing?

(Elissa) That number is increasing.
We've been doing a lot of work, again,

with classes. We tell teachers about the project,
they work with their students.

I do a webinar showing them how to use
the project, where we tutor and support.

And the classroom then give support at the
back end of the stage and research it.

So that number is going to increase. Again,
next week when I got another fourty students

from GW on this site. We do have the occasional
user who comes across it and then

goes hog while on it. That, as people find this
on their own, they would usually spend a lot more

time on it.
(Host) And how many followers do you have

on your @twitter name?
(Elissa) You mean personally or the museum?

(Host) Well @museums365 is that it?
(Elissa) That's - I forgot - about 1400.

The museum itself has 150,700 something.
(Host) So you do have an audience

that you can reach by that twitter feed.
So you use that for advertising events,

do you promote these citizen history projects?
(Elissa) We do, and particularly now, the way

that our social media team is set up,
I came over last October, and then

by a month behind me, we have analyst 
person come over from collections.

I've been in education for -- until the two of us
I ran the Lodz Ghetto project,

he ran Remember Me, which is
a crowdsourcing project in the band

of the American History Project
where we have people sharing their

personal knowledge, where the memories
of, judgement of who these persons.

We have photographs that we show, these children,
and ask "Does anybody remember this person?

Do you know who this person is?"
And people do and they share their story.

It's really been remarkable to see how successful
that's been. So we have two people working

within this crowdsourcing field, now sitting in 
the social media. And I'm very excited to see

what we can actually do with that, 
once we get out of the 20th muck.

(Host) I have many more questions but I should
let others, pursue.

(Audience) So, um, looking at the Children
of the Lodz Ghetto site, and right at the top

there's project status, so, twenty students
know to have survived, so is this what's been known

or verified through people working on this site?
(Elissa) That's right, yeah.

This is one of the additions that we put in
with the new revision of the site.

We had a bigger evaluation with some 
of our users, and a little bit work from

the administrators doing research, and they 
gave us some of their feedback.

Among that was, people want to see the scope
of what they are doing. How far along

we're actually getting with this project.
(Host) I think that's really important,

you've been including the number of 
citizen historians who have contributed

to the project. I think that's a good thing 
to show too. They do this at NYPL, show

the number of people, number of records
that have been curated or transcribed.

(Elissa) It's one of the things that they mentioned
in that same article about galaxy zoo,

was that, here at the three in the morning
with my galaxies, you know, there are

a couple thousand other people also
up at three in the morning with their galaxies.

So when their best [ribow] in the end,
where we're so often on our own,

we're actually very much with other people
at the same time.

I am an educator, I love questions, 
and I love wasting time, so as long as it takes.

(Audience) Yeah I didn't mention that I went
to the museum last week, and saw a tour

saying about this, I don't remember that 
there was anything, lets say, in the area

that talks about it. And I thought that, you know,
that might be a good thing,

to have something, where they're from
or something, where they go to talk about

this project, because, you know, 
looking around there are maybe,

I think, you know, elderly people who have
person of interest as they go to that museum.

That might open up more --
(Host) So it's like how does

the brick and mortar interact more tightly
with the virtual here.

(Elissa) And what we've been more willing
to do in the brick-and-mortar space is then to say

connect with us online. We've also been
missing a lot of our community museums

around the the symposium near some of our mall.
Where we'd get to the end of the exhibition

and say "What did you think? Tell us on twitter at
Am-History Museum." So we are more willing

to let people tell us, share their thoughts
in the social space. So putting things

in our Facebook wall, talking to them
on Twitter, putting videos on Youtube,

pinning stuff on Pinterest boards.
But as far as interaction with our

digital space, the things that are connected to us
in visual and outside of social,

we definitely have less of a push, 
to those into the museum itself.

There is a space on the second floor
of the museum, where our third

crowdsourcing project, we have three 
going on right now, to a very much end.

The World Memory Project,
we're in partnership with Ancestry.com

we have a bunch of names list, that we're
trying to get transcribed, and we open

those up to the Ancestry community to help
us key in some of those names and dates

and things from these giant databases.
And there are two stations that are set up

there. Where you are getting to help 
key in -- but again we don't talk about

it very much. And I often do wonder
if there is some kind of force separation

between our brick-and-mortar self,
and our digital space self.

Because the brick and mortar, we can 
control, pretty much. We can control

what conversations going in that space,
we have information comes down from

the museum at large. And the digital space
was a little bit scarier. Right? We're not

be able to control the conversations there
as much. We are worried that people

would just take our stuff and run away with it.
And if we are not ready for that many people

to look at our data and actually poke our stuff,
poke our precious raw files, then having

information leading to those things in
the brick and mortar space can be

a little bit scary sometimes.
(Host) And it's like on some of your tickets

it could say "Work with us online."
(Elissa) Totally.

I would love that.
(Host) So the museum's greatest fear

might be something like success where
people demanded more and more.

What's your biggest fear about citizen
historian projects in the Holocaust museum?

(Elissa) I think my fear is I'm going to fail.
And I believe in failing big and failing best.

But I am worried that when we build it
nobody will come, where we build it,

people come, and then we can't share
that with our internal community.

We say "Look at all these great success 
we had." And they say "So what.

What's the point?" That discrete
experiment we were running

where we have the trust of our users,
we have a wonderful community

that well iterates and talks to each other
and learn skills, and goes out into the world

that nobody on our side, or wasn't enough.
And that if this experiment fails,

then how are we every going to
convince them again?

(Host) It makes me think of -- there's
all this talk about blended online education,

and moves and the counter-discourse 
from people in pedagogy is about, well,

we need learning outcomes that
would be assessed. How do you measure

the education that you are giving?
But it seems to me that's the other

part of the Circle that we don't have
closed ear yet. It's -- how do we document

that we have taught citizens 
how to do history in a way that relives

our own sense of what it means
to do history. When we show how

many people -- we could show
how many people might have transcribed

something, how do we document
what they learned, and, argue me

the counter-argument to people
who say "So what? So you've got some people

who type some stuff in, big deal."
(Elissa) It's really hard, it's where I think

having the notes filled so prominent 
really comes in. That we've given

people the space, we ask them
to share with us what their reflections are.

And anecdotally I can tell you that
people, let's say, spend more and more time

on the project get better and better
at filling their skill, and they'd able to

reflect more critically what is it
that they are thinking. But in terms

of being able to measure, to give it
an A, I don't know if we can.

I don't have to figure that out yet.
We also have a lot of supporting

teachers, who haven't quite grasp
the idea either, I have one teacher

who wanted to use the project such
that the students would go on

and research one student, and they
would present the powerpoint

of that student's life, in class.
Then I had to tell him that

you can't do that, because you are
going to have kids who would go and

look for a student, and find nothing there.
That's the nature of the project,

that's the nature of doing research.
We don't know what we don't have.

And in finding that out, that's part of
the goal for us is to figure out

we don't have what those gaps are.
And so trying to put up a hearing

narrative on it, you can't always do that.
The expectations just aren't the same.

(Host) Now thinking about you using
the Bloom's model, you were saying that

as we think through what we want
to give people who interact with us,

we want to climb up the scale.
So, a kind of outcomes assessment

would be to somehow to map back
to that. And say, "We've brought people

from here to here to here.
But how would you make that assessment

as is, we need indicative and strictly 
from our own projects that are

trying to do this, so, self-interested
in an answer to this problem it seems.

Really hard stuff.
(Elissa) I imagine you have, like an

another crowdsourced group of people
who would then go through those

free text responses and code those.
So you would have something like a

separate project going on at the same time
where they'll be able to have certain words

that comes today we're looking for.
In those notes.

(Audience) I know that there's been
some discussion about this in the archives

field, particularly question of instruction
and how much when you bring in a group

of students into the archives and you
teach them how to do research,

teaching them actually handling the skills,
and what they've been doing

is a pre- and post-test. And trying 
to compare the results to see

what they've actually learned.
But there's a whole new set of research

that is going into this because no one 
is really quite sure that actually works.

But, I think this is a critical issue
for a lot of disciplines right now,

is trying to figure out what it is
you are trying to evaluate

and how you going to do that evaluation.

(Host) I'm wondering off, also it gets
to the top of the Bloom's pyramid ,

when you get to that true creative level,
but when you start seeing your users

able to take the skills that they acquired
in the course of the interaction with

the institution and create truly new
and different things, and the institution

has to be willing to accept that,
as almost like, well here's one of our

user's exhibit. You might even call it
an exhibit on this topic. It's their

interpretation, we don't necessarily
endorse it, but maybe when we give them

the space, the digital space in order
to demonstrate that creativity.

So they kind of move up from being
worker bees to, you know,

making something.
(Elissa) Should they take it even further

trusting now apart, to be able to --
(Host) Right, you know, way out there

interpretations, or people do stuff
with your data that you don't even like.

(Audience) And with the Holocaust Museum
you could imagine how that could go.

(Audience) One of the best ways to,
at least to being to get a sense of what

people are getting out of this 
is simply to ask them "What did

you get out of it?"
And if they are able to express that

in a way that is convincing, then you know
that it worked.

(Elissa) That's a good point. We have
one teacher, so that the teachers

going to be working with us starting
next week, again, it's been our

biggest fan for most of the time
the project's been up.

Who assigns students at the end of class
due two reflection papers.

One just the real basics of what did you find
on this day, how much time did you spend

on that project, what did you write,
what did the museum write back.

And reflect on that encounter.
And then a new page on just,

their experience of the site.
What it is that they, were thinking

about getting out of it,
what we could do better,

what they could do better.
Next topic. And I think,

in aggregate, that is the best we've been
able to do so far, being able to see what it is

that people are taking away from the project.
I think that if there is some way

to make that more of the part of the project,
to ask people as they leave this thing,

share something. Answers, questions
someone open with it, with us.

That we're kind of unsure.

(Audience) I don't know that much about
the -- really, a merging discipline

of learning outcome assessment,
knowing we have our own specialist

gathered through campus, but it's a lot
more complex than just asking people

what they think they've gotten out of it.
That's a part of it. And I really think

that we need to know and we need to
figure out ways to know what we are doing.

Because how can we know if, you know,
we're doing a good job of teaching

the things we want to teach through
these slides and through these participation.

How can we know how to change?
To better realize our goals.

Those are really complex issues 
and I am actually thinking out of,

trying to reach out to some learning
outcome assessment people just,

to help think through that part of the equation.
(Host) So I want to return to encourage you

to go much further with this, you know,
idea of printing it on the tickets

or making visible in the museum, 
and lots of other ways if you have

150,000 Twitter followers, you should 
be generating a lot more than 1500

participants. I mean, we work here
at the Smithsonian's Encyclopedia of Life project,

to make a webpage for every species,
and they have some of the same concerns

that you have, but I think you have a grand
opportunity to go to your wards and

your sponsors and rank this up 
as the central way. This is the future

of this museum. It's a matter of creating
out. That's one thing about educating

the users but, creating out, reaching and 
engagement in getting people to

participate remotely, that may generate
more traffic with people who

come and visit, there's just a lot of ways
this should grow bigger, and I'm,

you know, you should be shy of
that growing-ness much larger.

The fears are prevalent everywhere
and maybe the Holocaust museum

deservedly, as I said, I worked for them
on their -- and their fears are prevalent

about Holocaust deniers taking over these,
or polluting results. Even one small error

in the data set will then trigger a national
history that undermines the validity of it all,

so you do have more concerns than usual,
but all of the more reasons to go at it,

in an unsubstituted way, and deal with
the credibility of, you know, ensuring

the credibility so, it's good that you've got
the, sort of, senior reviewer status,

but various forms of badges and recognition
having annual conference for those

who participating, bringing them in,
bringing them together, raising your stature,

making them leaders of the project,
giving them decision making power

and supervision to control any problems.
There's lots of ways you can go much further

and demanding more of your users 
while actually causing them to engage.

So don't be afraid about that. 
I have one particular question about

the 1500. You have some distribution
of the demographics, I mean there's

two theories. One says that, well,
the museum patrons and interests

are of an older demographic, and 
the other says, well, it's the kids who

are doing online citizen science,
so help me with that one.

(Elissa) Well it's not about askewed,
but there's again, a lot of our outreach's

been through teachers, so, most
users here are school-aged,

so my best users have been in middleschool.
Which is for our middle-school educators

has been incredibly ratifying. 
But as far as empowering users,

people who find us not through a school,
just on their own, and then, crank out

at the data, they for the most part
been in college or just out of college.

(Host) I mean you could do a lot more,
I am a supporter, I am a contributor

and a member at -- I have no idea 
about the Lodz Ghetto project.

It's just not advertised, doesn't reach
me, in either the email traffic I get from

USAHMM or the printed materials,
or the annual reports or anything

that I get, so I mean I think there's a way
that you should be less shy, you should be

more bold in making these projects 
are more visible. That will raise the issue

of credibility but also the value 
to the museum and you need to

buy in to the people upstairs.
Your directors and your boards.

To be able to be in to this.
I mean, a memorable day was --

I was working and writing plan
for computers in this museum

where the 70 members of the Holocaust
memorial board, many spoke up against it

saying things like, "If the Nazis had computers,
you know, etc." So it was [Shanky Wineburg]

who was, sort of, the lead designer
of this, who said, I mean, settled it all

with a very sharp quote, he said
"Computers are the best way for

the next generation to learn about the Holocaust."
And it was over. You know, making that

forcible statement, that this is,
I'm glad to help you, if that would be useful.

I'm writing you an email, so
you'd be on with that, you know,

I think there's a lot that you can
and should be doing and revving up

internally as well as externally,
absolutely the way to go.

(Audience) I think what's interesting
is that if you trust your users enough,

say Holocaust deniers did get, well,
some material, I mean, how do you

teach people to do this area well?
History is all about refuting, argument

sets that don't hold up and learning
how to do that, and understanding

that those arguments will inevitably
crop up all the time, and as you raise

your profile you will get more of that.
So be prepared, but go there.

(Audience) Maybe the analogy to look at
with the cranks and support is,

is open source software community.
They're, by opening up the software,

you have a better chance of creating
something that is robust, and

it's going to be protected then if you
try to keep it to yourself, control it.

(Audience) I was thinking, while we're 
planning follow-up projects where you

<i>Laughter</i>

You mentioned that the audience
for this is still predominantly American.

Partly imagine that's because of working
with classes, but I wonder whether

there is a kind of pen-pal-asque kind of
angle to this idea on the internet,

that everywhere and you know,
the descendants of many of the people,

or people who might know about this,
or have other sources of information

are obviously probably still in, might still
be in Europe, or in Israel or wherever.

And I wonder about, sort of, a global
outreach, sort of, piece, and how that

fits in with the museum's position,
versus the other Holocaust and remembrance

institutions.
(Elissa) My interns actually are working on

German language arts program,
she's coming to us from Berlin this year.

She was totally jazzed about the Lodz
Ghetto project, and probably are

our heaviest moderator at the moment.
And I should send my boss a review,

as a German language outreach program,
to German schools, based on the things

in their curriculum, and be able to --
We had a good group of teachers

from Poland who came in last year.
And I was asked to come and present

the project to them. And there's actually
a lot of hesitancy about it, that

they didn't like the concept or the framework.
Except one woman who actually was

from Lodz, and she said it was a brilliant
idea and that her students would love

to work on it. Part of the problem is
that our resources are in English,

and all the data is in German. 
So we have to go through and say that

yes, Nona is name and Folun is it's 
first name. And through the expectation

for our English students, so there's 
a German name cheatsheet.

And for our German speakers we already
got the data at their disposal

and a lot of them are taught English
in schools. I'm less familiar with how other --

I guess we could view it as just English
class project, for schools. But I think

it's an excellent idea that we've paired
this with our global outreach since part

of this project still send some admission
called the global classroom

where we do talk about outreach
to the world.

(Audience) I'm curious about the Polish 
teachers' hesitancy.

(Elissa) Um, it was bad. Yeah, they didn't 
like the way we are posing our questions.

The fact that we just open these students
up for anybody to come and look at them.

And I think there's also some hesitancy about
the way that we are reading history.

Again the idea that history is, being
spanned out, is open. They weren't

[inaudible] the framework,
that there could be new questions

coming out of them. And that's your
families -- I'm sure it's not just

the polish teacher mindset
that it was a different group

with different questions.
And I definitely imagine that

when we are working with different
group of teachers and have different outcome.

(Host) If there are no other questions
or comments, let's have a round of applause

for a really great presentation.

<i>Applause</i>

Does not count as genuine.
The allographic work, by contrast,

such as a musical score or poem
has no one acceptable instance.

Or as Goodman puts it, all correct
performances or renditions of the work

are equally genuine instances o f it.
Allographic art, therefore we may

thereby define as a rule-bound.
Pondering the question, Goodman asks,

"Could institution of a notational system
transform painting or acting from

an autographic, into an allographic art."
Well Goodman answers the question

in the negative. "The development of
time-based media suggest that

we reconsider the issue. Past the work
of art in the digital era, become akin

to a symphony or a publication."
Does the aim of curators, conservators,

technical specialist and artists to sort out
the implications of such questions going forward.

As we consider the ramifications of time-based
art, which can be reproduced and decimated

outside the realm of traditional museum
environments, what is the significance.

of showing such work in museums, 
in a laminar institutions to become repositories

for such work. When might it be appropriate
to recognize that a work of art is essentially

ephemeral. And when and why might we want
to take steps to preserve it and perhaps

to transform it in order to preserve it.
To do so, ultimately, is to privilege

the idea over matter, recognizing that
we must inevitably allow the medium

in which the work was originally executed
to evolve, in the service of its presentation.

The opportunity to collect exhibit and
preserve time-based art, thus provides

an exceptional opportunity to consider
the philosophical locations of new media

for understanding our world and our selves.
As well as to explore the technical

and intellectual challenges of preserving 
these works for future audiences,

and for providing access to them,
for audiences now and tomorrow.

The new technological environment
produced by digital media further

privileges the value of interdisciplinary
and interinstitutional collaboration,

as we explore the tools and strategies
necessarily to share time-based and

digital works with future generations.
And on that note, I thank you so much

for your attention. And I very much looking
forward to hearing your thoughts, observations

and questions. Thank you.

<i>Applause</i>

(Anne) Yes
(Audience) First of all, I have a critical

question to ask, first of all let me give a --
thanking you for that extraordinary

presentation. I don't get to introduce myself
as I was away. I'm sorry about that,

but I'm coming to mid presentations
for years now as a fellow here.

This is one of the most remarkable
that I've seen. There's a lot of deeper

respect behind these questions.
My question is this: on the note of

[Benjamin] and he's sort of,
who was a figure that I distrust,

as someone was, as far as this type
goes as well, and he's mentoring

notions of the subject rendering
management of flux. I wanted to

get you to reflect on the fact --
there's a brave fascination in your idea

of the time series, and the various
flooring that go on with it.

You can get the point about how 
conventional ways of formulating

subjectivity are under attack.
But it strikes me as paradoxical

that the portrait library would be
this place where this radical project

would be going on, and before I want
to do that, would rather -- first of all

it seems to me that a lot of these
radical experiment that you put forward

are actually predicated just as much
as [Benjamin]'s essays of [inaudible]

I have a really nostalgic impulse to recover
the subject in the first place.

When I see those three late night 
talk show hosts, I was shocked by the news.

This is I think what the lips are supposed
to feel, that the identity of it all,

the fact that there were, makes me
long for a world that is better than that.

It's a reflection of my alienating world
that I want to see the individual,

so there's the nostalgia there.
But I think the problem is even greater

than that in my mind, that I constant
to engage in this radical project while

presuming that the subject is going 
to be a portrait, is to presume the very

thing that was the problem in the 
first place, you know what I mean?

Like, if I can put it, it's like the idea
of presuming the individual subjects

so that to attack that idea, is stacking --
is not a radical project in the first place.

When I put Lebron James all by himself
in a cube and evacuate the entire cube

of everything in the world except
images of himself and then conduct

a radical decentering from that,
I pre-supposed in the first place

in totally artificial terms, one, I'm presuming
that radically to attack.

There's something about this project 
going on in the space of the portrait gallery

that seems to presume the erratic enemy
in the first place, I just wanted put --

(Anne) I think it's a fabulous -- I think
It's a really really fabulous set of observations

that you put forward and I thank you
so much for that, and I have to say

one of the things that I love so much about
[Benjamin] and it's like any great author,

something that keeps me coming back
over and over is there are so many facets

obviously to all of his essays. I have to
admit the work of art in the age of mechanical

reproduction is this magnet for me.
And I'm just -- I put it obvious, I think you're

right, that he seems to be in many instances
sort of battling with his own sense of nostalgia.

And I will also say that I think I really do
consider his work extraordinarily artful.

It's obviously very self conscious in it's 
construction, as is the case with the artworks

I shared with you today. And so I guess
first and foremost I would say

I don't think there's any one way to read
any of these, and that ultimately

is the fascination. There are lots of 
different context in which these can

function. I do think that the work of art
in the age of mechanical reproduction

itself in terms of observations about
subjectivity is really really interesting,

particularly [Lee] in this essay, when he's
grappling with this question of Victorial

cliffs for example, and really dealing with
the fragmentation of the body,

and new ways in which we could get to
literally see and understand the body of result,

and freeze-frames it and photographic interventions
on, but that's a little bit of an aside.

And you bring up the really important question
of, alright, if I'm working at the portrait

gallery, how can I -- I notices it's not directed
personally, but how can one who is attached

to this notion of a portrait gallery in the
first place presume to undermine this notion

of an individual hand-on one of the things
that is important to understand about

the notion of the portrait gallery itself.
I don't mean just ours, but this larger

intellectual framework, as of course,
it too has, a history that relates to

a specific set of political developments,
and specific set of intellectual developments.

It is a product of mid nineteen century,
it seems to be a very British concept,

which is interesting, [Norship Pointing]
for example, has made the point that

portrait galleries tend to exist in the
English speaking world, which I actually

have come to think is attached to ways
of thinking about the political significance

of the individual unit in society
that is kind of interesting, especially

with respect to democratic ideals
so I have to say actually, I think there's

something really interesting about the 
perhaps hidden political assumptions

that go along with the portrait itself.
But specifically with respect to trying

to undermine and retask this initial
portrait gallery, that has a lot to do

with the fact that that's where I happen
to find myself as a young curator.

I ended up at the portrait gallery 
somewhat unexpectedly shortly after

finishing graduate school. And I --
one other things that really intrigued

me about it, and this is going back
twelve years, is that the museum

underwent a very self-conscious 
reinvention between 2001 and 2006

when it was actually under physical renovation.
And there was a desire to re-examine

the very principles of portraiture, which
I think has tended to be a form of art making

that has not gotten a significant amount
of credit, I think in the recent past,

it's been seen as a somewhat tired genre,
in fact, in the sixties lots of artists refuse

to use that term, we think of Chuck Close 
for example, who does these giant faces.

But during the sixties he called them heads.
He would not acknowledge until relatively

recently that they are a form of portraiture.
And so one of my pleasures, pleasures

perhaps as a curator has been to ask
audience to reconsider what they think

they know about portraiture by thinking
of it -- and this is a thorny term, I'm using

that word, but I wanted to do is to undo
the notion of portraiture and to recast it

a little bit as a way of thinking about
identity and breaking down personal identity.

But I think you are right to bring up
the question about whether or not

there are in fact some, you know, some 
types of paradoxes or some assumptions

that are invented in there that are,
you know, in some sense, going against

the grain of the deeper thinking here.
It is really interesting to me to talk with

contemporary artists and, actually,
a project I'm working on right now

is about portrait extraction, who really
do very actively seem to be rediscovering

or re-examining a notion which certainly
goes back to the Renaissance and this is

the notion that somehow in depicting 
anybody else, or anything else,

an artist is obviously reflecting something
of who he or she is, but I think the idea

that that entity can somehow be seen
as an envelope, that is impervious to

outside influence is really completely 
disintegrated. And yet side by side with that

we know that we live in this incredible 
culture of celebrity, and of course

[Worhose] was critiquing , so there 
definitely I think it's a very very very intersting

push-pull and I think you are right
to raise these questions on --

So I'm not sure that's a very satisfying 
response.

(Audience) I just wanted to underscore that
all these paradoxes that that you unintendedly

fly by underscore the interest of these lines.
Because it seems to me to speak to the

contradiction of the world that we live in.
So thank you very much.

(Anne) Oh, thank you. Thank for your
wonderful question.

(Audience) Hi, um, thank you for having us,
your talk was interesting.

I was wondering if the distinction of [inaudible]
autographic and allographic artwork

can really be helpful for preservation,
to artworks, because I think

the distinction is not that evident or --
there's more of a learning space between

the two, and I think they really applies to
all the media that is - all the work so far,

they are not necessarily time-based.
For example, sculpture by Turner,

and the way that it has to be reorganized
in the gallery according to certain

instructions because it travels in pieces,
but it has to be organized. You see,

that, in a way of performance, all the work,
because if something goes wrong,

you don't know where the things are,
you could argue that you are creating

a new work if you do that. So that means
if the first time that that was done

by the artist himself, that was the autograph
and is lost or maybe preserved through

photography. So that work is un-autographic
but it also has autographic instances.

And then it becomes untruthful work
so that if I show you a music as well,

in a sense you can have performances
in terms of someone performing the work

for, someone creating a new addition, 
but there will always be someone

that goes in before, the autographic
instances are very in, manuscript, for example.

And if we think [inaudible]
they exist in time-based media,

because you will look for it in each of the page,
you will look for proof of the first instance

of these sequence of art manifestations 
that will be steadily generated by the artists.

So, where's about that option
[inaudible] for preservation.

(Anne) That's a really interesting point.
I guess the assumption that you make

that there will always a desire to go
back to the original form of the

time-based piece, I think it's not
necessarily something that you should in fact

be taking for granted. It's actually
something, of course I have really

great colleagues, but it is a discussion
that I had with members of our staff.

Why do we need to hold on to this
original form, and again, this is where

I think the paradigm of being about 
being a historian is so important.

That my colleagues in exhibitions
department were more focused on

the here and the now, and getting it up
on the wall, for them, it's sort of,

excess baggage to worry about
the sixteen iterations that perceive it

it's not meaningful in the same way
in that context as it is to me.

I think they understand the value of 
preserving it, and ultimately I think that

that's where the framework of the museum
maybe have something special to

contribute to this dialogue, but this
distinction between allographic

and autographic I agree, is not a perfect one.
And in fact I think there are ways in which

intentions that we observe in the world of
time-based and digital media

are in fact really simply shedding light
on old problems that have always

been there. Our conservation, has always
been about intervention into, you know,

so-called erratic original, and 
the conservator has to make choices

about how to best represent the intent
of the original artist or at least what

is understood as being the original intent.
And what I really wanted to do with that

distinction was to, I guess, disengage from
the idea that there is some inherent,

well, of, but I as a historian I do think
there are things to be learned from

the original that may not even be
interesting to the artist, however,

that aside, I wanted to make a point that
if we begin to re-conceptualize visual art,

which is traditionally been seen as something
which is the product of an erratic genius.

You know, [Benjamin] is obviously trying
to disengage that, but it's sort of,

[fidelization] that continues, that we can
begin to see these works of art as things

that can migrate and retain some resemblance
of authenticity, no matter what medium

they are executed in, as long as they visually
represent or conceptually represent

what the artist wanted that piece to be,
but I do think it's an imperfect metaphor.

Things are going to change, things are
going to deteriorate and something ultimately

maybe a representation of itself.
And that becomes, I think it's almost

sort of interesting philosophical conundrum,
and I'll just say one more thing.

Which is simply to observe that this notion
of authenticity also functions

slightly differently for people who are
interested in preserving data,

and making sure that the data itself
doesn't get corrupted. So in fact,

I think that lots of interesting layers
get added in here, that are worth

thinking about, but it's a great question.
Thank you.

(Audience) I wanted to point out that
the idea of the essential self which

would be captured in the portrait is rather
a naive notion or is at fault with the public

presentation of a person. Everybody knows
these people have private lives.

Everybody knows they did all sorts of things,
they were complex beings. And if you take

something like -- well, it doesn't take
new media to bring out the complications

in the first place. You know, the diaries 
them-self are worth one avenue,

but the other thing is, photographic,
presentation as in for instance,

David Duncan spoke on Picasso
the private Picasso, he has this big

photographic record of Picasso 
in the fifties, the forties and fifties,

and you get this much complication.
In fact, you get a whole lot more complications

there than you can get in your average
presentation, well, you know, the one

of [Gitzburg], for instance. You get 
as much from David Duncan as you do

from the new media presentation.
And digitization doesn't actually change

anything so it's not quite that our notion
of a person's identity is modified by

the exposure of new media. The exposure
of new media is interesting if it's own right.

But it doesn't change the basic concepts
that we have of who we are,

what persons are, what vulnerabilities 
and complications we have.

(Anne) I think that's such a great
observation and would be so much fun

to dig into that question with you,
I would submit, I would like for the sake

of argument maybe put forward the idea
that I really do think there are ways

in which we are developing new insights
in the present day about self on which

perhaps are giving us new tools 
to go back and look at the past.

For example, the querying of the history
of art, for example. Not necessarily,

which is not to say that things were not
present previously that complicates

the picture, I think you are absolutely
right that there's always been

complexity with the human self.
But it is interesting to go back

and look at the language that
the artists use at least, in describing

their projects. Even somebody like
Alfred Stieglitz who was such

a perceptive and sophisticated photographer,
really looked for the essential moment

to capture somebody. And it's a language
but there's somehow I think, embedded

in that presumption of a privileged way
of understanding somebody. And yet of course

he did lots of different portraits of O'keeffe,
you can look at that series of portrait

presentations.
(Audience) I would not trust what an artist

says about his own project. It just isn't reliable.
It is self-promotional and --

(Anne) There's a narrative-reflective
paradigm but I loved -- I think your point

is an excellent one. I think you are 
pervasing it.

(Host) We have time for two more,
and there's a few people who have been waiting.

So one there and then at the back.

(Audience) Dealing with authenticity,
how, whenever you are deciding

to migrate or provide forms for
current exhibition, how do you deal

with deterioration versus intent.
For example, in [Globagrew]

the artist manipulated the signal 
to get different colors and distortion.

How do you know what's genuine
and how do you know what's real?

Especially with film, if it's a color film
and there's red shift, was that intended?

(Anne) Yeah, you know, the weird thing
is that you don't always know, actually.

There's a great piece at the [Hershorn]
by John -- no not John Jordan, um,

oh goodness, actually the artist's name
has just slipped my mind. But I'll get it

for you. There's this great film piece
by a very interesting artist who was

working in the seventies which is a film piece,
and there is sound that goes with it.

But there's a little bit of a hypothesis,
about how we think the artist wanted

that particular piece to be installed.
And the problem is there's an absence

of documentation. So actually, 
one of the things that's really interesting

and this goes to, really actually, any
period of artwork that we really have to

rely very heavily upon an interpretive
framework. And so one other thing

we've been doing in terms of looking
at this question about some practices

is to think about what it means to
document the intention of the artist,

at the outside. And so for example 
what we try to document now,

recognizing that this information can
very very quickly disappear, is, you know,

how does the artist want the piece to look
what it -- look when it's installed.

What is it supposed to sound like,
and of course inevitably even when

one tried to document these things
meticulously, we have to recognize that

there's inevitably going to be some slippage.
Even when you think you are being very

meticulous, things like processing
times, for computers can change.

And so I have to say that we do our best
to develop data that gives us as many

points of reference as possible, 
but I think ultimately we have to recognize

that it is to a certain degree,
an imperfect science. We also something

called a Checksum value to try to 
determine that the data moving forward

is kept in tack, but I think it's very
interesting that historically the --

in order to be sure that there are it,
problems for example, with the migration

of video into digital format, except
there's been curators, I mean,

[conservators], or probably curators too,
and certainly conservators who sit and look

intently at something to be sure that
there are no disruptions. We can't do that

with a generative work, so we've moved
beyond the point at which human perception

can really answer these questions for us.
And so I think on a certain level we have to

accept a certain degree of slippage,
and a certain degree of imperfection,

inability to completely nail something down,
and again, that is kind of a mind shift.

We've become comfortable with the fact
that we know everything will always be

something of an observation.
So I don't know if that --

(Audience) Those helped. Thank you.
(Host) So I'm afraid that we are out of time,

I'm sure Anne will be happy to stick around
if there are a couple of more questions,

but let's thank her for a really interesting clip.

<i>Applause</i>

(Anne) I can definitely stick around.
(Audience) What is a generative?

(Anne) Oh right, we started with this term of --
yeah, it's a relatively new term and it refers

to artwork that has no -- that doesn't loop.
That is continuously changing, so there is

code behind the image that leads to
ever-changing permutations of the way

in which the digital data is combined
and output. So there is no one instance

of the work. It's constantly changing.
One can describe the generative is --

(Audience) So a network piece,
is generative enough? It can

run on for a hundred years?
(Anne) Forever. And you'll see

ever-changing combinations.
(Audience) Yeah, maybe not very

interestingly different, but none the less.
(Anne) Yeah that's right, exactly.

You could just -- you did a pretty good job
describing it. Especially after fifty

or so minutes. Yeah.