Linear Algebra for Machine Learning

The TensorFlow channel on YouTube recently uploaded a video I made on some elementary ideas from linear algebra and how they're used in machine learning (ML). It's a very nontechnical introduction — more of a bird's-eye view of some basic concepts and standard applications — with the simple goal of whetting the viewer's appetite to learn more.

I've decided to share it here, too, in case it may be of interest to anyone!

I imagine the content here might be helpful for undergraduate students who are in their first exposure to linear algebra and/or to ML, or for anyone else who's new to the topic and wants to get an idea for what it is and some ways it's used.

The video covers three basic concepts — vectors and matrix factorizations and eigenvectors/eigenvalues — and explains a few ways these concepts arise in ML — namely, as data representations, to find vector embeddings, and for dimensionality reduction techniques, respectively.


Read More →

Warming Up to Enriched Category Theory, Part 2

Let's jump right in to where we left off in part 1 of our warm-up to enriched category theory. If you'll recall from last time, we saw that the set of truth values $\{0, 1\}$ and the unit interval $[0,1]$ and the nonnegative extended reals $[0,\infty]$ were not just sets but actually preorders and hence categories. We also hinted at the idea that a "category enriched over" one of these preorders (whatever that means — we hadn't defined it yet!) looks something like a collection of objects $X,Y,\ldots$ where there is at most one arrow between any pair $X$ and $Y$, and where that arrow can further be "decorated with" —or simply replaced bya number from one of those three exemplary preorders.

With that background in mind, my goal in today's article is to say exactly what a category enriched over a preorder is. The formal definition — and the intuition behind it — will then pave the way for the notion of a category enriched over an arbitrary (and sufficiently nice) category, not just a preorder.

En route to this goal, it will help to make a couple of opening remarks.

Two things to think about.

First, take a closer look at the picture on the right. I've written "$\text{hom}(X,Y)$" in quotation marks because the notation $\text{hom}(-,-)$ is often used for a set of morphisms in ordinary category theory. But the  point of this discussion is that we're not just interested in sets! So we should use better notation: let's refer to the number associated to a pair of objects $XY$ and $Y$ as $\mathcal{C}(X,Y)$, where the letter "$\mathcal{C}$" reminds us there's an (enriched) $\mathcal{C}$ategory being investigated.

Second, for the theory to work out nicely, it turns out that preorders need a little more added to them.

Read More →

Warming Up to Enriched Category Theory, Part 1

It's no secret that I like category theory. It's a common theme on this blog, and it provides a nice lens through which to view old ideas in new ways — and to view new ideas in new ways! Speaking of new ideas, my coauthors and I are planning to upload a new paper on the arXiv soon. I've really enjoyed the work and can't wait to share it with you. But first, you'll have to know a little something about enriched category theory. (And before that, you'll have to know something about ordinary category theory... here's an intro!) So that's what I'd like to introduce today.

A warm up, if you will.

What is enriched category theory?

As the name suggests, it's like a "richer" version of category theory, and it all starts with a simple observation. (Get your category theory hats on, people. We're jumping right in!)

In a category, you have some objects and some arrows between them, thought of as relationships between those objects. Now in the formal definition of a category, we usually ask for a set's worth of morphisms between any two objects, say $X$ and $Y$. You'll typically hear something like, "The hom set $\text{hom}(X,Y)$ bla bla...."

Now here's the thing. Quite often in mathematics, the set $\text{hom}(X,Y)$ may not just be a set. It could, for instance, be a set equipped with extra structure. You already know lots of examples. Let's think about about linear algebra, for a moment.

Read More →