On Neural Persistence, Improved Language Models, and Narrative Complexity

Introduction There is an amazing paper I got to read last week by [Rieck19] on the subject of persistent homology. The idea is that by borrowing ideas from topological data science, we can construct a per layer complexity metric. This complexity metric can then shed light into how generalized our learned representation is. Namely if …

On the Embedding of Narratives, and How it Pertains to Computational Neuroscience

Introduction A common question for the last article I posted was “Why did you reference [Riemann17]? Aren’t narrative embeddings identical to sentence embeddings?” For this post, I will consider narrative embeddings w.r.t. the ROCStories dataset. Firstly, we need to precisely define narrative embeddings. For this we need a few definitions. Fabula vs. Syuzhet Lets clarify …

Topology and You: What the Future of NLP has to do with Algebraic Topology

Introduction The size of NLP datasets has grown exponentially over the last few years. It was only a handful of years ago that 100-dimension (non-contextualized) word2vec [Mikolov et al.] embeddings were cutting edge. By comparison BERT embeddings are 700+ dimensions, Ai2’s ELMo is close to 1000 dimensions, and GloVe (the current industry standard) is typically …