WEEK #11

I want to record a 360 video and a normal video of the experience. To do so, I tried found a way to use my phone as a VR headset using Vridge from Riftcat. I also found a plugin for UE4 that allows me to record a 360 footage out of the player character. And, recently I borrowed a game controller from David. I will try to use all that to record my what the player sees, and I will pilot it.

Here is a screenshot from the VR interface.

WEEK #10

Next week, we had to send draft of our thesis. BLOGS ARE USEFUL!

If you are a student and read this, reporting what you do, how and why in your blogs is so useful for the thesis. I just had to copy paste all my previous report and the thesis was almost done. 

Last week I applied with this project to an open call from the gallery B39 at Bucheon for the exhibition Prectxe: Showcase Vol.2 and I had the chance to be selected. It will be held in June. I will probably know the exact schedule next week.

So now I have to prepare for the exhibition.

I was finally able to implement subtitles to my project but I had to get rid of the Doppler effect causing time stretch on the audio and desynchronization of the subtitles.

Subtitle Capture

Visualization of the voices' position.

WEEK #9

I organized the 3D space, giving a dystopian deconstructed architectures feel. The king Sejong at the center of the set. I connected all the elements to him with a lines. To give more details and refinement, I added bounding cubes around the scans and a small title reference.

The memories sounds is placed randomly and is binaural, so the audience can look for it. When it gets very close to one source, it can perceive a dot. This help finding the audio source and get the full sound loudness and understandability by getting closer to it.

WEEK #7 & #8

PART  I

It took me a week to prepare all the scans for Unreal Engine 4. Once done, I imported them into the scene. I had to change some lighting settings to make the objects display better.

I then tried to compile the project. It was useful to see if there were any bugs, if it wasn't too heavy for the computer or if the quality was good. 

I was satisfied of the overall performances and look.

PART II

It took me another week to fully develop the sound atmosphere of the piece.

Out of all the memories I got, I had to organize and shorten them, I ended up with a total of 35 testimonies.

I was fortunate enough to have a friend who helped me arrange the Korean versions. I've then sent these texts to the voice to speech Naver API which returned me voices.

The next stage was to create the music, edit the samples and add the voices. I've chosen to also create an audio only version of this project that is listenable on SoundCloud.

A capture of the timeline of the Ableton project.

Here a link to the script of all the memories : https://docs.google.com/document/d/1lQZP6Te2GfTwuWjNR2rrw96MGkDrRXBJbeYk8rM_V2Y/edit

WEEK #6

PART I

Since the beginning I thought about including crosses in my project. When we walk around, they are everywhere, they are constellations on the rooftop line of Seoul.

Unfortunately, it's quite difficult to 3D scan them, because their shape is thin, it would imply renting a drone one more time and flying in forbidden areas. 

So I decided to model one (for now) in Rhino. I will then probably texturise it using Substance Painter.

I used blender to retopologize it and add some displacement to make it more realist.

PART II

I will develop my virtual project on Unreal Engine 4.
Before a 3D scanned object is ready to import in Unreal Engine, there are important steps to follow.

I will explain my workflow from Metashape to Unreal Engine.
 
It's not good to directly use the 3D scan from Metashape because the mesh is too heavy (1.6M faces). So, we export the mesh from Metashape to ZBrush.

ZBrush has one of the best retopology algorithm. We retopologize the mesh and the face count drop from 1.6 million to 400,000 faces.
To avoid losing too much information, we project the high polygon mesh onto the new low polygon mesh. Thus the geometry of the new mesh is deformed to match better the old one's geometry.

We then export back to Metashape.

Back in Metashape we project the cameras on the new imported low polygon mesh in order to get the texture. Depending on the size of the mesh we create multiple textures. For this mesh I created 2 texture maps.

And we are ready to send it to Substance Painter.

Here, we enhance the texture and create the Material ready to use in Unreal Engine. We create a Normal map and an Ambient Occlusion map (using the the high polygon mesh as reference).

However, the mesh is not properly oriented and scaled. We use blender to transform it. Then we export as an FBX.

And finally we can import everything into Unreal Engine (see all the files in "Content Browser").
But before getting something like in the viewer, we need to reconnect the mesh with the Materials and create LODs (Level Of Details). Unreal decimates the mesh in 3 different LODs. It helps lowering the number of faces displayed on the screen and thus improve the fluidity of the game.

As for the Materials, we create a Material Master from which every Material Instances will take reference. This process makes creating Materials easier because we don't need to create parameters for each Material.

As you can see I'm not using Ambiant Occlusion here. I think for my project I don't need it because the texture of my scans are not delighted.

PART III

We went around Jongno with a friend to interview people and try to get memories of their past.
We only got 4 people. Two of them spoke with us during a long time. We got a lot of different stories.
I will write what I've selected as the highlights of these conversation.


"I miss walking at the market while holding my mother's hand."

"I feel safe here, I think I will still live a good life in the future."

"I wish I went to the army earlier, I would have had more time to study and find a good job."
"Thanks to the army, I feel patriot."

"I always feel sick when I'm drunk but I drink often. When I'm drunk I keep swearing hoping it will free me from my dizziness."

"One day, I won the 3rd place of the lotto, it's maybe one of my biggest pride."


"It's been 60 years that I moved to Seoul to earn money, I had a business but then I lost everything but then I earned a lot"
"Now I don't worry about anything, my life is easy"
"At 24 I was at the army for 5 years, fighting against North Korea and China. When the war ended I left the army, I'm happy that it didn't cause me any physical problem."
"After the army, I was building roofs and I earned 300 000 Won every day."
"I'm not specially proud of anything. Nor I regret anything. Now I'm retired, life is ok."
"I can drink a lot of alcohol, I drank one bottle at lunch, one bottle, one, one, one."
"I only drink a good amount, I drink a lot when I'm with my friends."
"When I drink I don't have any special thought."
"I feel patriot because I don't have to worry about anything, I don't need help from my family, I have 2 daughters, I can walk around, drink with friends, I'm really happy. If I I could go back in the past I wouldn't change anything."
"I don't want to have more money, I'm satisfied with what I have. If you have more money, you worry more. I have 2 daughters and I bought a house for them"
"When I was young, at my hometown I was farming with my family, we mostly grew rice. At my time we harvested by hand, we didn't have machines before. My family had a big land, with a cow, pigs and 2 dogs. But my parents sold the cow for me to come to Seoul. Now I live in Sodaemun"
"I knew the time before the oil and even before the coal. I knew the time when the only source of energy was wood."
"I remember when Cheongyechon was only a river without the city around. Then it became the industrial center of Korea, and recently the river has been renovated."


"My life is ok, life is life... No actually,  I'm not really happy, I'm lonely but there is one thing I'm proud of, gambling, especially horse racing. From 22 until now, even if I loose, I give money to the country and I help those in need. I only gamble on state owned games. Thus the money directly goes to the health system and public services."
"Compare to other people earning money from investment, stock market, I think I'm much better."
"Maybe I missed the time of marrying and founding a family because of gambling but I'm happy."
"There is one life, I'm partially regretful partially happy. But I cannot get out of it."
"Before economy was better, if you worked 3 month you got 7 to 8 million won. I had a business, I was selling things. But sometimes in 1 day I could end up with only one bill of a thousand won. I that moment I regretted it."
"At 28 I quit drinking, I think it's worst than smoking. It makes you die so early. But once a month, a drink a bit with friends."
"If I was rich I would do business. One day, I had 300 million won."
"These days, I'm reselling dumped products (taengchoeli) from department stores near by subway stations, such as expensive wallets. However, it's going to be hard to survive like this so I have to get a better plan, but I still don't know."
"Maybe I would like to open a shop but I need starting money and these days there is a lot of competition, young people are smarter than us".
"There is something I miss from when I was younger, I had a dream."
"A dream of making a big business. I met somebody when I was younger, a business partner. But maybe he wasn't a reliable person."
"People you meet really decide your path."
"Throughout my history, I saw many people achieving some businesses that I also thought about but I couldn't do it".
" I could have make a business and make job for the people and contribute to society. But I fell into gambling."
"I met a lot of part timers from university falling into gambling by watching us."
"Me and my brothers used to work with our older brother from a different mother, I had 15 years different with this older brother."
"I only went to middle school. I wasn't paid when working with this older brother he exploited me and my 3 other brothers. I used to wake up at 4am and work until midnight at a seafood retail shop."

"From 15 until 20 I liked someone but I never told her, I couldn't really express my feelings, I kept them in my heart. Because of the surroundings and because I couldn't really approach her."
"Looking at the past from now, surroundings make a difference. I couldn't do some things I wanted to do."

"There wasn't any homeless in Seoul back then. But one day at around 11pm in front of Seoul station, there was one homeless person. He was my age. I wanted to help. And I brought this person at a hotel. I got a room for him.
We became friends.
At that moment I had 300 million won, I was ready to start a business, I wanted to start with him.
But then I fell into gambling because of him.
I lost everything.
I couldn't even start a small business."

"Since I turned 58, I stop meeting people, finally I realized that I should stand by myself and renounce to have a wife."
"There is only one life."



"These past days my conditions was very bad, I worked too much every day, I was stressed a lot so I popped a pill and I slept, slept, slept, during all the weekend".

"Today, I learnt that my best friend and my older brother are dating and apparently it's serious. Apparently she often comes to my parent's house and help. It feels weird, I don't like that. Maybe they will marry. We will have to stop being friends, she will become Agassi."

WEEK #5

I would like to get more memories of different people. I'm going to diversify my sources by asking questions to people on the street. I will try to target people who have a different background than the samples I already collected.

Here are the questions I have to ask:

-
What's your name ?
Where is your hometown ?
Why did you move to Seoul ?

Are you happy here ?
What's not good about living here ?

What is your biggest regret ?
What is your biggest pride ?

What do you think about when you drank too much soju ?

Would you say you're patriot ? Why ?

If you could go back in the past what would you change ?
If you were rich what would you do ?

What are you doing right now ?

What do you miss from your youth ?
What would you miss if you were young again ?
-

A local friend will help me talk to people.

I will record the answer with a microphone and get the script from the records.



Some recent scans

New inputs in my memory collection:

내가 운전하면서 느낀 점은 사람들이 너무 난폭하게 운전을 해. 사고날 뻔한 적이 한두번이 아니야.

My mother gave up her life to raise me and my brother. As a child, I could see my mother's depression. But we were best friends, and we are best friends now. If my mother dies, I'm going to die with her.

When I was five years old, I celebrated my parents' birthday by stacking chocolate pie like a mountain. How could you do such a cute thing?

I was soon well outcast. It wasn't much, but I took revenge on all the children who bullied me. Happy ending!

We first met our cat when she was two or three months old, and she was white but now she is yellow.

나는 약 20년전 상수에 살았어. 그리고 오래 살진 않았지만, 그 때 상수는 정말 평화롭고 활력이 있었지. 지금은 너무 많이 변한 것 같아 안타까워. 홍대도 마찬가지로.. 동네가 매력이 많이 없어졌어.

Back In my middle school and high school days.. I got a good grade without math. (actually math grade was fine, not perfect though) But I can't considered by my parents as a high grade student because of that.

WEEK #4

I experimented with sound design to get an idea of what kind of atmosphere I'd like to create for the sound part of my project.

Here are 3 tests:



The two first tracks are more finished than the last one yet more freestyle. However, the last one is closer to what I think I'll make for the soundtrack. The slowed down voices are pansori songs and the computer generated voice will later tell the memories that I gathered.

This are "2D tracks" but the idea is to make a 3D sound atmosphere. It will be composed from a "core" that won't be spatialized (same level everywhere) and from different sounds which will be spatialized and located in different places of the 3D world. These spatialized sounds will be placed either randomly or either according to a meaning. They will consist of memories as computer generated voices and maybe some sound design elements to help make this 3D environment more natural. These 3D sounds will be perceived differently according to the position of the audience. If he/she is closer to the source of one sound, it will be louder, also these sounds will be subject to Doppler effect (the pitch will change according to the movement speed of the audience).

I will compose this 3D soundscape with the tools available inside Unreal Engine 4.


This is a screen capture from the last test I made.

And here is a quick follow up of what I've scanned this week.