Adding spheres to a scene using ARKit

IMG_1CA961D76C9F-1
Just some spheres floating above my bed

Boilerplate

First we’ll want to set up the basic ARKit scene. All this is boilerplate provided if you create an ARKit project in XCode


//  ViewController.swift

import UIKit
import SceneKit
import ARKit

class ViewController: UIViewController, ARSCNViewDelegate {

    @IBOutlet var sceneView: ARSCNView!

    override func viewDidLoad() {
        super.viewDidLoad()

        // Set the view's delegate
        sceneView.delegate = self

        // Show statistics such as fps and timing information
        sceneView.showsStatistics = true

        // Create a new scene
        let scene = SCNScene()

        // Set the scene to the view
        sceneView.scene = scene
    }

    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)

        // Create a session configuration
        let configuration = ARWorldTrackingSessionConfiguration()
        configuration.planeDetection = .horizontal

        // Run the view's session
        sceneView.session.run(configuration)
    }

    override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)

        // Pause the view's session
        sceneView.session.pause()
    }
}

One nice thing to add when developing with ARKit is the debug point thingies:


// Goes in viewDidLoad()
sceneView.debugOptions = ARSCNDebugOptions.showFeaturePoints

Spheres

Now, we’ll create a separate sphere class to abstract away the SCNNodes which will contain the sphere:


//  Sphere.swift

import Foundation
import ARKit

class Sphere: SCNNode {

    static let radius: CGFloat = 0.01

    let sphereGeometry: SCNSphere

    // Required but unused
    required init?(coder aDecoder: NSCoder) {
        sphereGeometry = SCNSphere(radius: Sphere.radius)
        super.init(coder: aDecoder)
    }

   	// The real action happens here
    init(position: SCNVector3) {
        self.sphereGeometry = SCNSphere(radius: Sphere.radius)

        super.init()

        let sphereNode = SCNNode(geometry: self.sphereGeometry)
        sphereNode.position = position

        self.addChildNode(sphereNode)
    }

    func clear() {
        self.removeFromParentNode()
    }

}

Now, when we want to add a sphere, we can do something like this


    func addSphere(position: SCNVector3) {
        print("adding sphere at point: \(position)")
        let sphere: Sphere = Sphere(position: position)
        self.sceneView.scene.rootNode.addChildNode(sphere)
// if we keep an array of these babies, then calling
// sphere.clear() on each will remove them from the scene
spheres.append(sphere)
    }

Insert into scene

Now for the interseting bit. To calculate the position at which we’d like to place the sphere, we can take the position of the camera and add a transform in the direction the camera is facing. All this code could be executed in a button press handler, for instance.

I found this handy screenSpacePosition function in a stack overflow post, it’ll do the vector math addition along the z axis. The distance is negative because we want to go further into the scene.


    func sceneSpacePosition(inFrontOf node: SCNNode, atDistance distance: Float) -> SCNVector3 {
        let localPosition = SCNVector3(x: 0, y: 0, z: -distance)
        let scenePosition = node.convertPosition(localPosition, to: nil)
        // to: nil is automatically scene space
        return scenePosition
    }

then, in the button press handler:


if let cameraNode = self.sceneView.pointOfView {

	let distance: Float = 0.3 // Hardcoded depth
    let pos = sceneSpacePosition(inFrontOf: cameraNode, atDistance: distance)    

    addSphere(position: pos)
}

hitTest()

Another way to do this would be to try to add the object “into” the scene, at a depth which is the location in z space of the object in front of the camera. We can do this with a hit test:


if let cameraNode = self.sceneView.pointOfView {

	let width = sceneView.frame.size.width;
	let height = sceneView.frame.size.height;

	let distance = sceneView.hitTest(CGPoint(x: width, y: height), types: [.existingPlaneUsingExtent, .featurePoint]).first!.distance;
	let pos = sceneSpacePosition(inFrontOf: cameraNode, atDistance: Float(distance))

    addSphere(position: pos)
}

And that’s all for this post!

LSTM Adventures

Following this as a guide: http://machinelearningmastery.com/text-generation-lstm-recurrent-neural-networks-python-keras/

1: Started by training with the single-layer LSTM

model = keras.models.Sequential() 
model.add(kl.LSTM(256, input_shape=(X.shape[1], X.shape[2]))) 
model.add(kl.Dropout(0.2)) 
model.add(kl.Dense(y.shape[1], activation="softmax")) 
model.compile(loss="categorical_crossentropy", optimizer="adam")

results:

  • final loss after 20 epochs: 1.9054
  • time to train (approx): 1hr
  • mostly gibberish words, but word lengths and spacing look right!
  • can handle opening and closing quotation marks
  • sometimes gets ” ‘?’, said the . “

the mort of the sorts!’ she katter wept on, ‘and toene io the doer wo thin iire.’

‘io she mo tee toete of ther ’ou ’ould ’ou ’ould toe tealet ’our majesty,’ the match hare seid to tee jury, the was aoling to an the sooeo.

‘he d crust bi iele at all,’ said the mock turtle.

‘ie doersse toer miter ’hur paae,’ she mick turtle replied, ‘in was a little soiee an in whnl she firl th the kook an in oare

the rieet hor ane the was so tea korte of the sable, bnt the hodrt was no kently and shint oo the gan on the goor, and whnn hes lene the rueen sas so aeain, and whnn she gad been to fen ana tuiee oo thin shaee th the crrr asd then the was no toeeen at the winte tabbit wat she wiite rabbit wat she mitt of the gareen, and she woole tee whst her al iere at she could,

*note: I tried this again with a 512 unit single layer LSTM, the results were along these lines:

‘i con’t sein mo,’ said alice, ‘ho would ba au all aare an ierse then and what io the bane af in eine th the bare aadi in the cinrsn, and she was soit ion hor horo aerir the was oo the cane and the was oo the tanl oa teing the was soie in her hand

and saed to herself, ‘oh tou dane the boomer ’fth the semeen at the soiee tf the bareee and see seat in was note go the pine afd roe banl th the grore of the grureon,

Training time was slightly longer, and the results are not meaningfully different. One observation here is that training for 20 epochs might be too short…

2:  Added another LSTM layer

model = keras.models.Sequential()
model.add(kl.LSTM(256, input_shape=(X.shape[1], X.shape[2]), return_sequences=True))
model.add(kl.Dropout(0.2))
model.add(kl.LSTM(256))
model.add(kl.Dropout(0.2))
model.add(kl.Dense(y.shape[1], activation="softmax"))
model.compile(loss="categorical_crossentropy", optimizer="adam")

results:

  • final loss after 20 epochs: 1.5144
  • time to train (approx): 2hrs
  • seems to fall into loops
  • closer to ‘english’ words

and mowte it as all, and i dan see the that was a little boom as the soog of the sable it as all. and the pueen was a little boowle and the thing was she was a little boowle and the that was a little boowle and the that was a little boowse and the thing was the white rabbit say off and the thing was she was a little boowle and the that was a little boowle and the that was a little boowse and the thing was the white rabbit say off and the thing was she was a little boowle and the that was a little boowle and the that was a little boowse and the thing was the white rabbit say off and the thing was she was a little boowle and the that was a little boowle

3: Trying again with 3 LSTM layers

model = keras.models.Sequential()
model.add(kl.LSTM(256, input_shape=(X.shape[1], X.shape[2]), return_sequences=True))
model.add(kl.Dropout(0.2))
model.add(kl.LSTM(256, return_sequences=True))
model.add(kl.Dropout(0.2))
model.add(kl.LSTM(256))
model.add(kl.Dropout(0.2))
model.add(kl.Dense(y.shape[1], activation="softmax"))
model.compile(loss="categorical_crossentropy", optimizer="adam")

results:

  • final loss after 20 epochs (took much longer!):
  • time to train (approx): 2.5 hrs
  • real words, better sentence structure… but it’s also falling into pretty small loops.
  • Different seed text results in the first line being quite different but then quickly devolving into the same loop of the hatter and the mock turtle going back and forth.
at the way that the was to tee the court.

‘i should like the dormouse say “ said the king as he spoke. 
‘i don’t know what it moog and she beginning of the sea,’ the hatter went on, ‘i’ve sat a little sable. 
‘i don’t know what you con’t think it,’ said the mock turtle.

‘i don’t know what it moog and she beginning of the sea,’ the hatter went on, ‘i’ve sat a little sable. 
‘i don’t know what you con’t think it,’ said the mock turtle.

‘i don’t know what it moog and she beginning of the sea,’ the hatter went on, ‘i’ve sat a little sable. 
‘i don’t know what you con’t think it,’ said the mock turtle.

4: Single Layer GRU (for comparison with the single layer LSTM)

model = keras.models.Sequential()
model.add(kl.GRU(256, input_shape=(X.shape[1], X.shape[2])))
model.add(kl.Dropout(0.2))
model.add(kl.Dense(y.shape[1], activation="softmax"))
model.compile(loss="categorical_crossentropy", optimizer="adam")

results:

  • final loss (20 epochs): 1.87897 (approx the same as the 1 layer LSTM)
  • time to train: 40 minutes (faster!)

tone of the hoore of the house, and the whrt huon a little crrree the sabbit say an in oish of the was soiereing to be a gooa tu and gerd an all, and she whrt huow foonnge to teyerke to herself, ‘it was the tait wuinten thi woils oottle goos,’

‘hht ser what i cen’t tamling,’ said alice, ‘ih’ le yhit herseree to teye then.’

‘i shanl sht me wiitg,’ said alice,

‘what ie y sheu wasy loteln, and teed the porers oaser droneuse toe thet, and the tored harden wo cnd too fuor of the word the was soteig of the sintle the white rabbit, and the tored hard a little shrenge the fab foott of the was sotengi th theee th the was soe tabdit and ger fee toe pame oo her loot ou teie the was soee i rher to thi kittle gorr, and the tar so tae it the lad soe th the waited to seterker

Still gibberish, though more english words. Training for more than 20 epochs might be the key!

Here’s a graph of the loss dropoff over time

alice_loss.png

 

Personal Goodreads Ranking Method

This is how I rank books I’ve read.

[link to my goodreads account]

Goodreads uses a scoring system where you can give a book a number of stars out of 5. The usefulness of a scale like that is in using its full range, for instance if I were to rank all books 3, 4, or 5, the scale has really become a 3 point scale and has lost some resolution.

So I’ve been scoring books based off these semantics:

1 star: “Did not like it”. Really not a fan. Wouldn’t recommend, and would probably actively recommend against reading it. Some books have this score, which is surprising because it likely means that I finished reading it (and probably shouldn’t have). Either not worth the time I spent reading it, or made me upset for having wasted my own time.

2 stars: “It was okay”. Reasonable. Not life changing. Not going to recommend it. Pretty much totally neutral. Most recently I read a book about ‘tips for sleeping better’ which I gave a 2. It’s pulp, probably pretty good pulp, but pulp nonetheless.

3 stars: “Liked it”. Worth the read, probably wouldn’t recommend it readily. Most books get this score. High quality average book.

(4 and 5 I’d both call “favorite” or “top books”)

4 stars: “Really liked it”. Really good, I would recommend this to a friend looking for a read. This book will stick with me and become good intellectual fodder to chew over, or a story which I still think about. Probably if you know me you’ve heard me spout some knowledge or something from this book.

5 stars: “It was amazing”. 10/10, everyone should read this book. I’ve never read a book twice but I should really probably read this book twice. Or thrice. This score means the book has drastically changed my worldview, broadened my imagination irreversibly, and/or given me whole new language for thinking and talking about certain ideas.

One-Straw Revolution

 

This book is Zen and the Art Of Motorcycle Maintenance-esque, in that the first half is about this Japanese guy’s life work and discipline of running a self-sustaining farm, and the second graduates into his life philosophies (as farming has taught him) at length.

Of course, I haven’t ever read ZatAoMM, so I could be off on the comparison.

Fukuoka’s got a pretty small farm (by modern industrial farming standards), an acre for grains and veggies, and 5 or so acres for a mandarin orange orchard. People come to live on his farm to be disciples of his way of life, the farm sustains about 15 people at a time. It’s mostly self sufficient apart from a few purchased goods: soy sauce and vegetable oil, things which would be impracticle to self-produce, and I’m sure some amount of salt/spices/tools, etc are also purchased. They sell a mandarin oranges for profit.

Fukuoka was a research scientist in microbiology and agriculture before falling ill, and while in the hospital realizing that western agriculture was bullshit and that he was going to go do it differently.

He calls his practice do-nothing farming, his mentality is that letting the natural systems do their work means that the farmer can do less. He doesn’t use pesticide to kill insects eating the plants, which allows other insects higher up the food chain to stop the ones eating the plants, and other animals to eat those in turn. Avoiding pesticides allows the whole food chain to reach a happy fixed point, a dynamic equilibrium. Pesticides also have side effects of burning nutrients out of the soil over time. So letting nature run its course means that the farmer doesn’t have to spend the time applying pesticides, and the soil is allowed to grow strong as well. Win-win.

The numerical yield from his farm is the same or higher than that of comparable farms using much more labor-intensive processes.

His grain planting practice was particularly interesting, rice in the warm season and rye/barley in the cool. He doesn’t do any of the complex flooding and transplanting process usually associated with rice farming, instead scattering (he calls it ‘broadcasting’, which I quite like) rice seed in spring among the previous season’s rye and barley, harvesting the rye/barley and letting the rice grow, scattering rye seed, harvesting the rice and letting the rye/barley grow, and so on. All the while he also scatters clover seed, which forms a nice low base layer to help the soil retain moisture and help keep the seeds from being eaten by birds.

Fukuoka’s life philosophies are largely that humans have created a whole bunch of extra work for themselves (see pesticide example), which then begets more work to sustain the side effects, etc.

He says we’ve lost touch with what tastes good, with our natural ability to determine what foods are healthy and what foods our body needs at a given time. He says that really, food straight from the gardens simply prepared is truly delicious, but we’ve convinced ourselves that we need more complex preparations for food to be worthwhile.

I found the book in a list of permaculture/no-till farming authors, this certainly fits the bill. Last week I broadcasted some spinach seed in my vegetable patch in the backyard, as the most accessible version of an experiment of his technique. So we’ll see if the birds eat it or what.

Revenge of Geography

Great book to follow up on Accidental Superpower, slightly more academic and much more depth. Kaplan spends more time discussing history geopolitical understanding, as well as on deeper exploration into various regions around the world.

Starting with Central Europe, he works through the rest of Europe, the Eurasian Heartland, Russia, China and neighboring seas, Japan, Southeast Asia, India and the subcontinent, and the Middle East, finishing with an analysis of the US especially in regards to Mexico.

Indian Subcontinent

I was particularly interested in the tension in between India and Pakistan (given my family origins). The subcontinent has a more complex geography than it appears at first: India is broken up by horizontal rivers on a vertically organized peninsula, and further by highlands and mountain regions. This has prevented a unified India for much of its existence, despite partial consolidation of power in the north or south at various time periods. Kaplan states that historically, control of the subcontinent has stemmed from joint control of Delhi and Lahore. The formation of Pakistan is a geographical anomaly and a curiosity in that the border between the two nuclear nations has become hyper-militarized without any sort of true “geographical logic” as grounds. The steppe from Afghanistan (another almost-failed state whose mountainous geography defies central control) to northern India has no defined demarcation. To Kaplan, Pakistan represents the history of invasion of India from the northwest, poised on the border in nation-state form.

USA/Mexico

The US-Canada border is geographically logical: there is the intra-Canadian isolation affected by the Canadian Shield, dense forestation on the border, and the Canadian climate driving the population to be pushed up against the southern fringe. The US-Mexico border is not this way. Kaplan points out that the American tactic of investment in remote conflicts in the Middle-East is trivially misguided (or at least that continued investment will be) compared to investment in assisting in the development in Mexico. Militarized borders trend economically and sociopolitically in the direction of the less developed nation: the solution is that a more developed and stable Mexico will create joint prosperity in the US. Over time, Mexican immigration into the US will see a demographics shift (especially in the southwestern states) which has already begun, an increase in Latino and Spanish-speaking peoples which will influence the culture of the country.

Kaplan emphasizes the joint importance (and long-term prevalence) of geography with the actions of “Great People” to self-determine. He has no illusions that geography is not a predetermined fate — and also no illusions about ignoring geographical lessons in the hopes of a post-geography world.

Highly recommended.

Man’s Search For Meaning

Following my 2/3rds completion of The Gulag Archipelago, I selected to read this popular and highly acclaimed book by psychiatrist Viktor Frankl.

The first half is a tale of Frankl’s experience in the Nazi prison camps (including and mostly centering around his time at Auschwitz). He uses this basis of experience to justify and give depth to his understanding of human psychology: people in the camps were able to find meaning when they had nothing else. This meaning allowed them to live, he even states that meaning is a primary motivator in human Being, not a secondary effect of more ‘primal’ survival needs.

He shares, in the camps cigarettes were used as a commodity; they were hard to come by–sometimes as a reward for labor from the guards–, highly valued, not smoked, and traded for a bowl of soup and so on. When you saw a man smoking his cigarettes, you could be sure he was going to die soon: the man had given up his will to live, falling towards seeking immediate pleasure and comfort, and once lost the will was impossible to recover.

Frankl repeatedly references Nietzsche’s: “He who has a why to live for can bear almost any how.”

The second half of the book provides an overview of Frankl’s Logotherapy, a now-field of psychiatry focusing on a person’s ability to generate meaning for themself. Frankl explores the paradox of happiness and success. If one is explicitly seeking happiness (or success), the goal becomes harder to access. He says: Success cannot be pursued, it must ensue.

Ultimately, man should not ask what the meaning of his life is, but rather must recognize that it is he who is asked. In a word, each man is questioned by life; and he can only answer to life by answering for his own life; to life he can only respond by being responsible.

Liberty devolves into boredom and chaos without the twin pillar of Responsibility. Frankl even mentions off hand that there could be a Statue of Responsibility on the west coast to complement the Statue of Liberty on the east. The lack of understanding of responsibility is a major factor in our society’s current crisis of western existentialism.

Heuristic: He urges the reader to live through each situation as if we are going through it for the second time. Imagine that the first time, you chose the absolute worst possible course of action, and then act with this knowledge in mind.

So live as if you were living already for the second time and as if you had acted the first time as wrongly as you are about to act now!

The Gulag Archipelago

I started reading The Gulag Archipelago on recommendation from U of Toronto psychology professor Jordan Peterson. In his lectures he repeatedly references Gulag as a reminder of the cruelty capable by humans, a cruelty which lurks around the corner of our contemporary society and supposed civilized manner.

Gradually it was disclosed to me that the line separating good and evil passes not through states, nor between classes, nor between political parties either—but right through every human heart—and through all human hearts. This line shifts. Inside us, it oscillates with the years.

Gulag is about the soviet prison camps from the 1920s to the late 1950s. While the institution of the Gulag was technically shut down in 1960, Gulags in softer forms persisted all the way until the camp Perm-36 was closed in 1987.

This is recent history.

Arrests

Alexandr Solzhenitsyn discusses at length the random arrests which occurred before our author himself was arrested.

Arrests occurred systematically, in the middle of the night, and to no public outrcy. Arrests occurred for no specific reason— most were legally admitted under Article 58 of the Russian Penal Cod, which blanket-banned all “counter-revolutionary activities”. Arrests were most commonly handled by the arrestee with a meager “Who, me? What for?”. And then off to the holding camps.

Solzhenitzyn makes it clear that there’s no clear point at which the injustice occurring around ones-self becomes unbearable. There’s no signaling whistle to rally the people to take up arms, or to fight ‘for real’.

After reading, it is altogether too easy to imagine the same happening in the US (entirely irregardless of whether we’re under the rule of the rabid Right or the rabid Left).

Wrecking

The stalinist-communist regime specifically targeted the intelligentia, especially the engineering class in the society— being in control of the train systems, water and sewage systems, and so on. The engineers were constantly under suspicion of “wrecking”, that is, engineers destroying state property (and of course, state property is your property, you would do well to inform on any wreckers!). Engineers were put under surveillance to make sure they were not wasting resources in their construction and maintenance (the grand irony of course is in those 5 idle individuals monitoring the one productive person).

Prisoners

Solzhenitzyn discusses in great detail the life of the prisoner, the process of being moved between camps. The conditions of disease, starvation, rape, and so on to which the prisoners were subjected (as their sentences grew for reasons or non-reasons beyond rational comprehension).

Since then I have come to understand the truth of all the religions of the world: They struggle with the evil inside a human being (inside every human being). It is impossible to expel evil from the world in its entirety, but it is possible to constrict it within each person.

Thieves

He discusses the thieves in these camps— arguably the only members of the prison system who had done much to deserve their fate. The thieves were the upper class of the prisonfolk (below the guards of course, but close). According to Soviet Socialist logic, thieves were created as products of the corrupt society in which they were born, they are the true victims of our cursed non-communist society! If we had communism, the thieves would not have had to steal to provide for their poor selves. And they got away with whatever within the camps, to very little discrimination. Counter-communist thought was the true enemy.

Children

Children growing up in the Gulags of course grew into adaptation to their world, growing into horrible unsalvageable wretches who would steal in gangs from the frail and elderly, beat people for fun and so on. Solzhenitzyn relays one account of an older man who, when seeing an unsuspecting child, would sneak up and push the kid face down into the mud, pressing with his knee on their back until hearing their ribcage crack. Doctors would never be able to figure out what was wrong, and the child would die within a few months.

The Nuremberg Trials have to be regarded as one of the special achievements of the twentieth century: they killed the very idea of evil, though they killed very few of the people who had been infected with it… And if by the twenty-first century humanity has not yet blown itself up and has not suffocated itself—perhaps it is this direction that will triumph? Yes, and if it does not triumph—then all humanity’s history will have turned out to be an empty exercise in marking time, without the tiniest mite of meaning! Whither and to what end will we otherwise be moving? To beat the enemy over the head with a club—even cavemen knew that.

I am not the storyteller to do justice to much of what Solzhenitzyn has written.

I recommend this book to anyone interested in exactly what a totalitarian regime can transform into, especially regimes with fantasies of creating a better life for those who feel they do not have one.

If only there were evil people somewhere insidiously committing evil deeds, and it were necessary only to separate them from the rest of us and destroy them. But the line dividing good and evil cuts through the heart of every human being.

To do evil a human being must first of all believe that what he’s doing is good, or else that it’s a well-considered act in conformity with natural law. Fortunately, it is in the nature of the human being to seek a justification for his actions… Ideology—that is what gives evildoing its long-sought justification and gives the evildoer the necessary steadfastness and determination… Thanks to ideology, the twentieth century was fated to experience evildoing on a scale calculated in the millions.”