https://mattf1n.github.io <<<<<<< HEAD Thu, 26 Mar 2026 00:00:00 +0000 https://mattf1n.github.io/posts/2026-03-26-blocking-cisco-socket-filter-on-macos.html https://mattf1n.github.io/posts/2026-03-26-blocking-cisco-socket-filter-on-macos.html Thu, 26 Mar 2026 00:00:00 +0000 The Cisco Secure Client Socket Filter installed by the Cisco VPN client on my laptop (VPN is necessary to connect to USC compute cluster) was causing problems: sometimes when I started up my laptop I couldn’t access the internet even when…

]]> ======= Fri, 13 Mar 2026 00:00:00 +0000 https://mattf1n.github.io/posts/2026-03-13-my-research-ai-coding-workflow.html https://mattf1n.github.io/posts/2026-03-13-my-research-ai-coding-workflow.html Fri, 13 Mar 2026 00:00:00 +0000 My eyes generally glaze over reading “how I use AI” posts, but enough of my colleagues have expressed interest that I think this would be worth sharing. My overall strategy for coding these days is to have OpenAI’s Codex agent write and…

]]> >>>>>>> 1776c23a60a83e77d1042194aba24ff41895eda4 https://mattf1n.github.io/posts/2026-02-24-better-math-fonts-for-ai-venue-submissions.html https://mattf1n.github.io/posts/2026-02-24-better-math-fonts-for-ai-venue-submissions.html Tue, 24 Feb 2026 00:00:00 +0000 Most AI conference venues have Times (or Times New Roman) as their font for submissions. Unfortunately the math in the templates is set in computer modern (or latin modern, not sure). Improve your paper’s aesthetics by using Times for the…

]]> https://mattf1n.github.io/posts/2025-12-08-papers-that-caught-my-attention-neurips-2025.html https://mattf1n.github.io/posts/2025-12-08-papers-that-caught-my-attention-neurips-2025.html Mon, 08 Dec 2025 00:00:00 +0000 Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations shows that language models are fine with however you tokenize your doc. Can we hide information in the tokenization…

]]> https://mattf1n.github.io/posts/2025-04-24-lexicalization-questions.html https://mattf1n.github.io/posts/2025-04-24-lexicalization-questions.html Thu, 24 Apr 2025 00:00:00 +0000 Do langauge models “lexicalize” certain multi-token words or phrases (i.e., treat them as atomic units)? How would we measure lexicalization in LMs?

]]> https://mattf1n.github.io/posts/2025-04-22-lora-weight-decay-thoughts.html https://mattf1n.github.io/posts/2025-04-22-lora-weight-decay-thoughts.html Tue, 22 Apr 2025 00:00:00 +0000 Irhum makes an observation that I have thought about from time to time: LoRA-adapted model weights decay to the original model during training (not 0). How much of LoRA’s success could be attributed…

]]> https://mattf1n.github.io/posts/2025-04-04-recursive-reasoning-transformers.html https://mattf1n.github.io/posts/2025-04-04-recursive-reasoning-transformers.html Fri, 04 Apr 2025 00:00:00 +0000 ]]> https://mattf1n.github.io/posts/2025-04-03-llm-have-built-in-macs.html https://mattf1n.github.io/posts/2025-04-03-llm-have-built-in-macs.html Thu, 03 Apr 2025 00:00:00 +0000 pico.sh seems like a cool, minimal effort tool for blogging, but it doesn’t support math, so I’m going to roll my own.

]]> https://mattf1n.github.io/posts/2024-12-17-bug-in-torch-code.html https://mattf1n.github.io/posts/2024-12-17-bug-in-torch-code.html Tue, 17 Dec 2024 00:00:00 +0000 Torch code bug that took me an hour to fix: I wrapped a function in @torch.inference_mode which called another function which called another function that was trying to call torch.backward.

]]> https://mattf1n.github.io/posts/2024-12-05-hello-world-mathml-netnewswire.html https://mattf1n.github.io/posts/2024-12-05-hello-world-mathml-netnewswire.html Thu, 05 Dec 2024 00:00:00 +0000 I created an RSS feed for my website. I did it by hand, this template. I don’t use any frameworks for maintaining my website, I just write everything by hand. I don’t change my website often…

]]> https://mattf1n.github.io/posts/2024-10-21-right-way-to-ensemble-language-models.html https://mattf1n.github.io/posts/2024-10-21-right-way-to-ensemble-language-models.html Mon, 21 Oct 2024 00:00:00 +0000 Suppose you have n langauge models with embedding size d, vocabulary size v, and softmax matrices W₁, W₂, …, W_n ∈ ℝ^v × d and you want to sample from them as an ensemble. One…

]]> https://mattf1n.github.io/posts/2024-10-21-research-interest-demo.html https://mattf1n.github.io/posts/2024-10-21-research-interest-demo.html Mon, 21 Oct 2024 00:00:00 +0000 I have created a discord server for people interested in collaborating with the lab. Email me for an invite!

]]> https://mattf1n.github.io/posts/2024-10-21-obtaining-logprobs-from-llm-api.html https://mattf1n.github.io/posts/2024-10-21-obtaining-logprobs-from-llm-api.html Mon, 21 Oct 2024 00:00:00 +0000 Many LLM APIs give top-k logprobs in their outputs. What if we want to obtain all the logprobs? Here I present two algorithms for obtaining logprobs from an LLM API. Both of these depend on the API allowing us to add a logit bias to…

]]> https://mattf1n.github.io/posts/2024-10-21-differentiable-binary-to-onehot.html https://mattf1n.github.io/posts/2024-10-21-differentiable-binary-to-onehot.html Mon, 21 Oct 2024 00:00:00 +0000 I would like to define a differentiable function f : {0, 1}^log v → {0, 1}^v that converts binary number representations of log v bits into one-hot vectors. This can be accomplished by using fuzzy logic operators to convert…

]]> https://mattf1n.github.io/posts/2024-10-21-deep-ba-sampling-extending-bat.html https://mattf1n.github.io/posts/2024-10-21-deep-ba-sampling-extending-bat.html Mon, 21 Oct 2024 00:00:00 +0000 TL;DR: we can use any intermediate LM representation to prove that a subset of next-token candidates have non-zero probability.

]]> https://mattf1n.github.io/posts/2025-12-12-heavy-tails-and-diversity-in-model-distributions.html https://mattf1n.github.io/posts/2025-12-12-heavy-tails-and-diversity-in-model-distributions.html Tue, 12 Dec 2023 00:00:00 +0000 Direct sampling from model output distributions often gives incoherent outputs. Some have attributed this to a heavy tail, i.e., the model assigns too much probability to low-probability tokens. My goal is to test this hypothesis.

]]> https://mattf1n.github.io/posts/2023-10-21-visualizations-gallery.html https://mattf1n.github.io/posts/2023-10-21-visualizations-gallery.html Sat, 21 Oct 2023 00:00:00 +0000 These are some visualizations I have made over the years, both for academia and for fun!

]]> https://mattf1n.github.io/posts/2023-10-21-softmax-function-is-linear.html https://mattf1n.github.io/posts/2023-10-21-softmax-function-is-linear.html Sat, 21 Oct 2023 00:00:00 +0000 I asked my Twitter followers if they knew that the softmax function is linear. The result was disbelief.

]]> https://mattf1n.github.io/posts/2023-03-21-ss-py-semantic-scholar-cli.html https://mattf1n.github.io/posts/2023-03-21-ss-py-semantic-scholar-cli.html Tue, 21 Mar 2023 00:00:00 +0000 ss.py is my personal command line tool for searching and citing academic papers via the Semantic Scholar API. About page. GitHub.

]]> https://mattf1n.github.io/posts/2022-11-28-word-count-in-vim-statusline.html https://mattf1n.github.io/posts/2022-11-28-word-count-in-vim-statusline.html Mon, 28 Nov 2022 00:00:00 +0000 This ftplugin updates the word count in the statusline on every save. More frequent updates slow Vim down and cause random rendering problems.

]]> https://mattf1n.github.io/posts/2022-10-11-configuring-zathura.html https://mattf1n.github.io/posts/2022-10-11-configuring-zathura.html Tue, 11 Oct 2022 00:00:00 +0000 I finally got Zathura (the pdf viewer) configured the way I want it on MacOS. I installed using homebrew following these instructions. I set up an automator script to launch Zathura for me…

]]> https://mattf1n.github.io/posts/2021-01-20-washington-state-relief-map.html https://mattf1n.github.io/posts/2021-01-20-washington-state-relief-map.html Wed, 20 Jan 2021 00:00:00 +0000 I learned some Blender and used some open source elevation data to make a nice looking relief map of Washington State. Check out how I made it here. :round_pushpin:

]]> https://mattf1n.github.io/posts/2020-05-14-camping-in-cottonwood-wash.html https://mattf1n.github.io/posts/2020-05-14-camping-in-cottonwood-wash.html Thu, 14 May 2020 00:00:00 +0000 This last weekend, Caitlyn and I backpacked up Cottonwood Wash in Utah’s San Rafael Swell, a beautiful canyon all to ourselves. We even spotted some petroglyphs. :mountain:

]]> https://mattf1n.github.io/posts/2020-04-20-course-notes-from-cs183.html https://mattf1n.github.io/posts/2020-04-20-course-notes-from-cs183.html Mon, 20 Apr 2020 00:00:00 +0000 These are my notes from the course CS183: Foundations of Machine Learning. They are imperfect and incomplete but I really enjoyed making them. If you would like to make edits [email…

]]>