My internship at Khan Academy
I spent the past summer working on the iOS team at Khan Academy.
Khan Academy is best known for their videos, but there are also hundreds of interactive math exercises that students can use to practice skills ranging from counting to calculus. This is what an exercise looks like on the website:
For the past few months, the mobile team has been working hard to bring these exercises to the iPad app. Here are some of the parts I helped build...
Most exercises have related videos that students can watch if they're stuck. My first project was to implement a native version of the related videos interface. We liked how the YouTube app lets you drag down a video to interact with other content, so that's what I started with.
So we played with having three states: hidden, maximized, and docked. The docked video would play the video in the corner, letting the student interact with the exercise while watching the video.
I spent a lot of time playing with these prototypes. I noticed that if your finger wasn't centered in the video view when docking the video, the finger would become detached from the video as it shrunk, ruining the illusion of direct manipulation. I fixed this by setting the view's anchor point to the touch position, which makes the view shrink around the user's finger—a simple change that prevents limb-view detachment. See for yourself:
Plot twist! Most of this didn't make it into the app. When we tested with students, it wasn't clear that the fancy interactions were helpful. So we ended up shipping a simpler implementation without docking.
My next project was to make a native version of the exercise scratchpad. A lot of students use the scratchpad on the website to do rough work, even though writing with a mouse can be painful, so we knew it was a valuable feature, and that we could provide a far better experience on a touch device. This ended up being my biggest project.
It was decided early on during the development of the app that exercise content would be rendered in a web view—it was the only feasible way to bring hundreds of exercises with the resources we had. This meant the scratchpad canvas (a native view) would cover the webview rendering the exercise.
This arrangement was complicated by the fact that all exercises have interactive regions. For example, in the exercise below, the blocks have to be dragged to put them in the correct order.
So the main problem was hit testing: when a touch starts, should we start drawing on the scratchpad or let the webview handle the touch?
After an initially-promising but ultimately sketchy solution that
relied on internal implementation details of how native touches
on the touch event, it became clear that the right way to do it would
be to ask the webpage directly.
From the native side, when a touch begins on the scratchpad canvas,
interactive element. At first, I did a simple synchronous call
this didn't feel very good, especially on older iPads. The blocking
call would drop the first few points of the touch, which was
particularly annoying for short strokes, like decimal points.
As a nice side-effect, this async API is forward compatible with
WKWebKit (more on that in a bit).
There were a few other interesting problems to solve with the scratchpad that I might write about later. Big thanks to Andy for improving inking and a beautiful refactor, and Matt for helping with two-finger scrolling.
I got to work on a bunch of smaller things as well. Here are some of them.
I took a break from iOS land during the 4-day Healthy Hackathon and teamed up with Andy and Ben to work on a proof-of-concept for a UI prototyping tool. Having written way too much code to explore different interactions for the related videos, we wanted to explore how we could make iterating on interactive designs faster.
Our editor lets you create and place elements visually to set the scene, and only write code to describe the interactions. This hybrid approach saves a lot of boilerplate code.
UIWebView → WKWebView
was announced, I spent some time trying
to make the app use it on iOS 8+ devices, while continuing
UIWebView on iOS 7 devices.
This involved converting all the communication between the app and webview to
be asynchronous, since
WKWebView's API for calling
But when I actually ran the app on an iOS 8 device, nothing loaded.
WKWebView can't load
files directly from the local filesystem (this is probably going to be fixed).
Just for fun, I hacked up a version of the app with a local webserver to
serve the local files. In brief unscientific testing (i.e. I just
played around with some exercises), I didn't notice huge performance improvements.
Launch to exercise
I added a way to go directly to a specific exercise when launching the app from Xcode. A simple change, but one that saved a lot of time, especially when debugging issues for a specific exercise.
I got to work on really fun problems with really great people, which are the two secret ingredients for a great internship. Thanks to everyone at Khan Academy, especially the iOS team for teaching me a ton: Mike Parker, Andy Matuschak, Laura Savino, and Marcos Ojeda.
And thank you for reading this. If you liked this post, follow me on Twitter! ❤
P.S. If you want other interns' experiences, here are a bunch of posts: