Adam TuttleThe digital garden of Adam Tuttle2024-01-05T13:30:00Zhttps://adamtuttle.codes/Adam Tuttleadamtuttlecodes@gmail.comWelcome to OSX, Evergreen Edition2024-01-05T13:30:00Zhttps://adamtuttle.codes/blog/2024/welcome-to-osx-evergreen-edition/<p>I had an old version of this article <a href="https://adamtuttle.codes/blog/2016/welcome-to-osx-2016-edition/">written in 2016 <span style="font-size: 50px">👴🏻</span></a>, but obviously a lot has changed since then. Instead of rewriting it every few years, I'm going to designate this as the "evergreen" version. I'll update it as I find new tools that I like, or as my workflow changes.</p>
<h2 id="fundamental-improvements%3A-finder%2C-terminal%2C-karabiner-elements" tabindex="-1">Fundamental Improvements: Finder, Terminal, Karabiner Elements</h2>
<p>It took me a while to not be annoyed by Finder. There were other things I used (<a href="http://totalfinder.binaryage.com/">TotalFinder</a>) or looked at from a distance (<a href="http://www.cocoatech.com/pathfinder/">Path Finder</a>), but after many years, vanialla Finder does the job.</p>
<p>To be honest, the default Terminal app is <em>fine</em>, and miles ahead of Windows CMD or PowerShell. But since when is <em>fine</em> good enough? You have a variety of good options to choose from on Mac. I'm currently using <a href="https://www.warp.dev/">Warp</a>, but if that wasn't an option I'd be using <a href="https://www.iterm2.com/">iTerm2</a>.</p>
<p>As crazy as it sounds, even inheriting an extra modifier key with OSX ("command" in addition to shift, control, and alt) I still occasionally find myself wishing for another. So I use <a href="https://pqrs.org/osx/karabiner/">Karabiner Elements</a> to map the CAPS_LOCK key to cmd+ctrl+alt+shift, which some people call "super."</p>
<p>I have <code>ctrl+\</code> configured as my terminal visor shortcut. I leave it running 24x7 and can use that keyboard shortcut to bring up a terminal window no matter what I'm doing.</p>
<h2 id="developer-stuff" tabindex="-1">Developer Stuff</h2>
<p>In my terminal, I prefer to use Zsh with <a href="http://ohmyz.sh/">Oh-my-zsh</a>. There are dozens of great plugins and even more themes to choose from.</p>
<p>I write my code in <a href="https://code.visualstudio.com/">VS Code</a>. For the most part I think I use the default keyboard shortucts, though I have added and modified a handful that I'll have to come back and add at some point. <a href="https://adamtuttle.codes/blog/2013/My-Sublime-Keymap-Common-KB-Shortcuts/">This was my sublime keymap</a>, and I probably ported some of that over. Extensions I use:</p>
<ul>
<li>Apache Conf</li>
<li>Auto Rename Tag</li>
<li>Better Comments</li>
<li>Bookmarks</li>
<li>CFML</li>
<li>Cloak</li>
<li>Disable Ligatures</li>
<li>Docker</li>
<li>DotENV</li>
<li>embrace</li>
<li>ESLint</li>
<li>FileUtils</li>
<li>GitHub Actions</li>
<li>GitHub Copilot</li>
<li>GitLens</li>
<li>Import Cost</li>
<li>JavaScript and TypeScript Nightly</li>
<li>Material Icon Theme</li>
<li>npm Intellisense</li>
<li>PostCSS Language Support</li>
<li>Prettier</li>
<li>Rainbow CSV</li>
<li>shadcn/svelte</li>
<li>sort lines</li>
<li>Svelte for VS Code</li>
<li>Svelte Intellisense</li>
<li>Tailwind CSS IntelliSense</li>
<li>Template String Converter</li>
<li>Text Pastry</li>
<li>Thunder Client</li>
<li>Todo Tree</li>
<li>Total Typescript</li>
<li>Twoslash Query Comments</li>
<li>Wrap Console Log</li>
<li>YAML</li>
</ul>
<p>I also write my blog posts in VSCode using Markdown. It's all <a href="https://github.com/atuttle/blog">hosted on GitHub</a>, actually. For managing MySQL databases (local and remote) I use <a href="https://github.com/Sequel-Ace/Sequel-Ace">Sequel Ace</a>.</p>
<p>Pretty much everything else that I use near-daily is <a href="https://nodejs.org/">Node.js</a> and node modules. If you just need to stand up a quick basic static-file web server in a random directory, I like <a href="https://github.com/knpwrs/nws">nws</a>.</p>
<p>Do enough Node stuff and eventually you'll run into native modules that require compiling locally on your system. That, or if you do any iOS development at all (even with PhoneGap), you're going to need XCode. Better to just bite the bullet early and install it / update it as needed through the App Store.</p>
<p>On the off chance that you need an (S)FTP/S3 client, <a href="https://panic.com/transmit/">Transmit</a> is pretty good.</p>
<p><a href="https://itunes.apple.com/us/app/microsoft-remote-desktop/id715768417?mt=12">Microsoft Remote Desktop</a> is actually reasonably good for managing a few windows boxes remotely, but <a href="https://royalapps.com/ts/mac/features">Royal TSX</a> is better.</p>
<p>Most of the time I do my Git work in the terminal, but occasionally I'll want a GUI for block-level staging or history browsing. In those cases I like <a href="https://www.sourcetreeapp.com/">SourceTree</a>.</p>
<p>I use Keynote for presentations where I won't be doing any live code demos, or various web presentation frameworks when I am.</p>
<h2 id="other-great-stuff" tabindex="-1">Other Great Stuff</h2>
<p>You <em>will not find a better password manager</em> than <a href="https://agilebits.com/onepassword">1Password</a>. Fast, secure, and beautiful to boot. Integrates really well with major browsers and has system keyboard shortcuts for quick access. They even have an Android Keyboard that makes accessing your passwords on the go a snap. I bought a family license and forced it on my wife and mom, too. When my kids are old enough to start having passwords for stuff, they'll be forced into it too.</p>
<p>All of the computers in my house backup to <a href="https://www.backblaze.com/">Backblaze</a>.</p>
<p>For email, I've tried a bunch over the years. None are ever as good as straight up webmail. That said, <em>all</em> of my email is through Google Mail. If you have a work Exchange server or something, I can understand why you would want a local native client. I just don't have a recommendation for you. Lately I've been using <a href="https://mimestream.com/">Mimestream</a> as a native mail client because it has good integration with gmail, and the least-awful dark mode I've seen.</p>
<p>I have access to somewhere between half a dozen and a dozen google calendars, and coordinating them can be a real pain. I really like <a href="https://cron.app/">Cron</a>.</p>
<p>For a basic running todo list I have been using the native apple Reminders app. It looks better on mobile than on desktop, but it syncs automatically and it has <em>most</em> of the features I want. I also dabble with notes in <a href="https://obsidian.md/">Obsidian</a> that border on to-dos, but when pressed to give my primary to-do source, it's Reminders.</p>
<p>I have used <a href="https://www.libreoffice.org/">LibreOffice</a> for office documents (word, excel, etc), but the modern apps from Microsoft aren't half bad. Whatever you do, <a href="http://www.theguardian.com/technology/askjack/2015/sep/03/switch-openoffice-libreoffice-or-microsoft-office">don't use OpenOffice</a>!</p>
<p>If I need to record some or all of my screen, I always use QuickTime Player (which should come on your Mac). I don't do a ton of video editing yet, so iMovie is still sufficient for my needs. I see <a href="http://www.apple.com/final-cut-pro/">FinalCut</a> in my future, though. Just like Windows, the best video player is <a href="http://www.videolan.org/vlc/index.html">VLC</a> hands down.</p>
<p>Need to figure out what's eating up so much disk space? Try <a href="https://nektony.com/disk-expert">Disk Space Analyzer</a>.</p>
The Case for Decorators in CFML2023-11-17T00:00:00Zhttps://adamtuttle.codes/blog/2023/the-case-for-decorators-in-cfml/<p class="photo-byline"><img src="https://adamtuttle.codes/img/2023/jake-melara-2k26mRosr2o-unsplash.jpg" alt="A man stands with his back to the camera in a scraggly field on a dreary gray day with his hands down at his sides, and a variety of papers hover in the air around him at jaunty angles" />
Photo by <a href="https://unsplash.com/@jakemelara?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Jake Melara</a> on <a href="https://unsplash.com/photos/man-standing-on-grass-2k26mRosr2o?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a></p>
<p>I work really hard not to have to write any CFML any more, but it always seems to claw its way back into my life.</p>
<p>I have a product in production (its 8 year go-live anniversary is tomorrow! 🍰) that uses CFML and the CFML ORM (based on Hibernate). The vast majority of this code was written 6+ years ago and continues to hum along nicely, doing its job just fine. So, when a request to add atomic-change auditing to certain data updates slid into my inbox, I braced myself and dove back into the code.</p>
<h2 id="what's-an-atomic-change%3F" tabindex="-1">What's an atomic change?</h2>
<p>Assume you have a database table (<code>Jouplers</code>) with many columns. Now assume that users occasionally update existing rows. Now assume that you want to look back at your audit logs and see that Jane Doe changed the value of the <code>splork</code> column from <code>fluargh</code> to <code>nerk</code> last Friday at 4:53pm.</p>
<p>The naïve approach is to log the entire record before and after the change in the audit log and let the person who needs to know what changed deal with diffing the two data blobs and figuring out what changed. That's not very kind, it increases the data storage cost, and also we were specifically asked for atomic change logging so I guess we should do that instead.</p>
<p>An atomic change is just the bits that changed. In the example above, the atomic change would be <code>(Jouplers #9417021) splork: fluargh -> nerk</code>. You can represent that a million different ways, but the point is that it only shows the information you need to understand what changed, rather than the entire contents of the record before and again after the change. It was the <code>Jouplers</code> table, record # <code>9417021</code>, and the <code>splork</code> column changed from <code>fluargh</code> to <code>nerk</code>.</p>
<h2 id="how-do-you-skin-that-cat%3F" tabindex="-1">How do you skin that cat?</h2>
<p>I considered a variety of different approaches. None of them felt right, mostly because they would all be verbose to use and require modifying a LOT of existing code.</p>
<p>Other than the approach that I will outline below, the one that came the closest to satisfying my requirements was to use database triggers to respond to the fact that the row changed and do the diff and logging in the handler. Setting aside for a moment the fact that even just the idea of DB triggers give me a fatal case of HELLLLL-NOOOOOO, we don't have any other triggers and I don't think anyone on my team has any recent experience working with them, so it would not be a good fit for our stack.</p>
<h2 id="my-solution" tabindex="-1">My solution</h2>
<p>At some point I remembered that CFML supports <code>onMissingMethod</code> in its components, and I could use (a bastardized version of) the decorator pattern to encapsulte an ORM entity, intercept invocations of its getters and setters, use that as an opportunity to check for changes and store them in memory in anticipation of a later request for the list of changes.</p>
<p>Before I show you the code for how the object works, let's first look at how you use it. Here's an example of how the code would look BEFORE adding my decorator:</p>
<pre class="language-js"><code class="language-js">transaction <span class="token punctuation">{</span><br /> <span class="token keyword">var</span> joupler <span class="token operator">=</span> <span class="token function">entityLoadByPK</span><span class="token punctuation">(</span><span class="token string">'Jouplers'</span><span class="token punctuation">,</span> <span class="token number">9417021</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> joupler<span class="token punctuation">.</span><span class="token function">setSplork</span><span class="token punctuation">(</span><span class="token string">'nerk'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token comment">//don't forget the other 75 columns that might</span><br /> <span class="token comment">//need to be updated here...</span><br /> <span class="token function">entitySave</span><span class="token punctuation">(</span>joupler<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>And after:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">transaction <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">var</span> __entity__ <span class="token operator">=</span> <span class="token function">entityLoadByPK</span><span class="token punctuation">(</span><span class="token string">'Jouplers'</span><span class="token punctuation">,</span> <span class="token number">9417021</span><span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">var</span> joupler <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">com<span class="token punctuation">.</span>atuttle<span class="token punctuation">.</span>AtomicORMEntity</span><span class="token punctuation">(</span>__entity__<span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> joupler<span class="token punctuation">.</span><span class="token function">setSplork</span><span class="token punctuation">(</span><span class="token string">'nerk'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token comment">//don't forget the other 75 columns that might</span></span><br /><span class="highlight-line"> <span class="token comment">//need to be updated here...</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token function">entitySave</span><span class="token punctuation">(</span>joupler<span class="token punctuation">.</span><span class="token function">__getEntity__</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token function">saveAuditLog</span><span class="token punctuation">(</span><span class="token operator">...</span><span class="token punctuation">,</span> joupler<span class="token punctuation">.</span><span class="token function">__getChanges__</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>What about this approach makes it appealing? The only modifications to existing code are to add the new wrapper around the entity once it's loaded (as close as we can get to a real "decorator"), and to get it back out of the entity when we're ready to save it.</p>
<p>This turns out to be the whole reason for this article. I talked about this project on <a href="https://workingcode.dev/">my podcast</a> and some of the listeners in the <a href="https://workingcode.dev/discord">podcast discord</a> got to discussing it, and it didn't make sense to give a full accounting of this code in discord and not share it here.</p>
<p>Other than those two things (and getting the changes for auditing, which would need to be done somehow anyway, so I'm not counting that), this is <em>entirely transparent</em> to the existing code. And remember, there are (many) dozens of setters being called, per entity. And this is a big app with many entities.</p>
<p>I got some guff in the podcast discord when I shared screen shots of this code in action, because apparently <a href="https://www.infoworld.com/article/2073723/why-getter-and-setter-methods-are-evil.html">getters and setters are evil</a> (link provided by guffers). 🤷🏻♂️ The code is the way it is, and work has to continue. I don't have time in my budget to build hydration methods that take an object of properties and hydrate the entity into each ORM entity in the project.</p>
<p>Build a time machine, then go back and take it up with the developers that wrote that code in 2016. And then do me a favor and get some of that McDonald's Szechuan sauce on your way back.</p>
<h2 id="how-it-works" tabindex="-1">How it works</h2>
<p>Here is a very basic, incomplete example that I think does a decent job of painting the picture for what we're trying to accomplish.</p>
<pre class="language-js"><code class="language-js">component <span class="token punctuation">{</span><br /><br /> variables<span class="token punctuation">.</span>entity <span class="token operator">=</span> <span class="token function">nullvalue</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> variables<span class="token punctuation">.</span>changes <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">function</span> <span class="token function">init</span><span class="token punctuation">(</span><span class="token parameter">entity</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> variables<span class="token punctuation">.</span>entity <span class="token operator">=</span> arguments<span class="token punctuation">.</span>entity<span class="token punctuation">;</span><br /> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">function</span> <span class="token function">onMissingMethod</span><span class="token punctuation">(</span><br /> <span class="token parameter">string missingMethodName<span class="token punctuation">,</span><br /> struct missingMethodArguments</span><br /> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">left</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>missingMethodName<span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">'set'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">var</span> propName <span class="token operator">=</span> <span class="token function">right</span><span class="token punctuation">(</span><br /> arguments<span class="token punctuation">.</span>missingMethodName<span class="token punctuation">,</span><br /> <span class="token function">len</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>missingMethodName<span class="token punctuation">)</span> <span class="token operator">-</span> <span class="token number">3</span><br /> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">var</span> newValue <span class="token operator">=</span> missingMethodArguments<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> <span class="token function">__set__</span><span class="token punctuation">(</span> propName<span class="token punctuation">,</span> newValue <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">invoke</span><span class="token punctuation">(</span><br /> variables<span class="token punctuation">.</span>entity<span class="token punctuation">,</span><br /> missingMethodName<span class="token punctuation">,</span><br /> missingMethodArguments<br /> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">__set__</span><span class="token punctuation">(</span><br /> <span class="token parameter">required string propName<span class="token punctuation">,</span><br /> newValue</span><br /> <span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">var</span> <span class="token function-variable function">getter</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">invoke</span><span class="token punctuation">(</span>variables<span class="token punctuation">.</span>entity<span class="token punctuation">,</span> <span class="token string">'get'</span> <span class="token operator">&</span> propName<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">var</span> <span class="token function-variable function">setter</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span> <span class="token parameter">x</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">invoke</span><span class="token punctuation">(</span><br /> variables<span class="token punctuation">.</span>entity<span class="token punctuation">,</span><br /> <span class="token string">'set'</span> <span class="token operator">&</span> propName<span class="token punctuation">,</span><br /> <span class="token punctuation">[</span>x<span class="token punctuation">]</span><br /> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">var</span> trackable <span class="token operator">=</span> <span class="token function">__isTrackableProp__</span><span class="token punctuation">(</span><br /> arguments<span class="token punctuation">.</span>propName<span class="token punctuation">,</span><br /> arguments<span class="token punctuation">.</span>newValue<br /> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>trackable<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">var</span> currentVal <span class="token operator">=</span> <span class="token function">getter</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span> currentVal <span class="token operator">!=</span> arguments<span class="token punctuation">.</span>newValue <span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> variables<span class="token punctuation">.</span>changes<span class="token punctuation">.</span><span class="token function">append</span><span class="token punctuation">(</span><span class="token punctuation">{</span><br /> <span class="token string-property property">'field'</span><span class="token operator">:</span> arguments<span class="token punctuation">.</span>propName<span class="token punctuation">,</span><br /> <span class="token string-property property">'from'</span><span class="token operator">:</span> currentVal<span class="token punctuation">,</span><br /> <span class="token string-property property">'to'</span><span class="token operator">:</span> arguments<span class="token punctuation">.</span>newValue<br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token function">setter</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>newValue<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">__isTrackableProp__</span><span class="token punctuation">(</span><br /> <span class="token parameter">required string propName<span class="token punctuation">,</span><br /> newValue</span><br /> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//if you're trying to SET an entity as a value,</span><br /> <span class="token comment">//we don't care to track that...</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">isObject</span><span class="token punctuation">(</span>newValue<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token comment">//if it's an array, that can only be useful to pass</span><br /> <span class="token comment">//an array of entities for a relationship</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">isArray</span><span class="token punctuation">(</span>newValue<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token comment">//if the current value is an entity, we don't</span><br /> <span class="token comment">//care to track that...</span><br /> <span class="token keyword">var</span> current <span class="token operator">=</span> <span class="token function">invoke</span><span class="token punctuation">(</span><br /> variables<span class="token punctuation">.</span>entity<span class="token punctuation">,</span><br /> <span class="token string">'get'</span> <span class="token operator">&</span> propName<br /> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">isObject</span><span class="token punctuation">(</span>current<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token comment">//if it's an array, that can only be useful to pass</span><br /> <span class="token comment">//an array of entities for a relationship</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">isArray</span><span class="token punctuation">(</span>current<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token comment">//otherwise, we should be able to track that...</span><br /> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">public</span> <span class="token keyword">function</span> <span class="token function">__getEntity__</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> variables<span class="token punctuation">.</span>entity<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">public</span> <span class="token keyword">function</span> <span class="token function">__getChanges__</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> variables<span class="token punctuation">.</span>changes<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>Still with me? Let me summarize the code above:</p>
<ul>
<li>Constructor accepts an ORM entity as an argument and stores it in a private variable</li>
<li><code>onMissingMethod</code> exists to intercept all methods being called on the entity. If it's a setter, we'll pass that to our custom <code>__set__</code> method that has the special sauce, else just proxy the request to the original method since it's not something we care about.</li>
<li><code>__set__</code> is where the magic happens. In addition to setting the value on the entity, it also checks to see if the value changed, and if so, stores the change in memory for later retrieval.</li>
<li><code>__getEntity__</code> returns the possibly-modified entity back to the calling code for saving.</li>
<li><code>__getChanges__</code> returns the list of changes that were tracked by the decorator, if any.</li>
</ul>
<p>There are a few other details that help make that all work, but at its core, that's what's going on.</p>
<p>As a reminder, the entire reason I'm writing this article is to explain why <code>__getEntity__()</code> is necessary to certain people. Hopefully the above code made the point.</p>
<p>But, we've come this far, and I cut quite a bit out of the component above (as well as adding lots of linebreaks to make it more easily readable in this format) in order to keep the length down. I'll go the last mile and include the entire content of the component below in case anyone wants to follow in my footsteps.</p>
<p>But before I do that, here's my final thought on the whole decorators thing.</p>
<h2 id="but-what-about-decorators%3F" tabindex="-1">But what about Decorators?</h2>
<p>As I mentioned, CFML doesn't currently have any first-class support for decorators. There are of course a variety of ways that they could be implemented, and I'm not interested in staring into that particular navel right now. But if we had decorators, how might this have all been different?</p>
<p>The only significant difference it would have made would be to remove the need for calling <code>joupler.__getEntity__()</code> in this line:</p>
<pre class="language-js"><code class="language-js"><span class="token function">entitySave</span><span class="token punctuation">(</span>joupler<span class="token punctuation">.</span><span class="token function">__getEntity__</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>A true decorator would be able to behave as if it were the object being decorated, which means it could be passed as the argument to <code>entitySave()</code> in this case:</p>
<pre class="language-js"><code class="language-js"><span class="token function">entitySave</span><span class="token punctuation">(</span>joupler<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//this is the decorated Joupler entity</span></code></pre>
<h2 id="the-full-code" tabindex="-1">The full code</h2>
<p>I'm not going to explain every difference between this and the shorter version above, other than to say this: The vast majority of the differences you'll notice stem from the fact that the concept of <code>null</code> is just an afterthought sort of stuck on to the side of the CFML language with some chewing gum and a bit of re-used duct tape. There's a lot of jumping through hoops in order to handle all of the various places you need to be thinking about nulls, and the various ways that nulls can be problematic.</p>
<p>Anyway, here's Wonderwall:</p>
<pre class="language-js"><code class="language-js">component <span class="token punctuation">{</span><br /><br /> variables<span class="token punctuation">.</span>entity <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span><br /> variables<span class="token punctuation">.</span>changes <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">function</span> <span class="token function">init</span><span class="token punctuation">(</span><span class="token parameter">entity</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> variables<span class="token punctuation">.</span>entity <span class="token operator">=</span> arguments<span class="token punctuation">.</span>entity<span class="token punctuation">;</span><br /> <span class="token comment">// writeDump(var=variables.entity, top=1);abort;</span><br /> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">function</span> <span class="token function">onMissingMethod</span><span class="token punctuation">(</span><span class="token parameter">string missingMethodName<span class="token punctuation">,</span> struct missingMethodArguments</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">left</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>missingMethodName<span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">'set'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span> <span class="token function">len</span><span class="token punctuation">(</span>missingMethodArguments<span class="token punctuation">)</span> <span class="token operator">></span> <span class="token number">1</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//not a "dumb"/generated setter, probably a custom method we added to the entity</span><br /> <span class="token keyword">return</span> <span class="token function">invoke</span><span class="token punctuation">(</span>variables<span class="token punctuation">.</span>entity<span class="token punctuation">,</span> arguments<span class="token punctuation">.</span>missingMethodName<span class="token punctuation">,</span> arguments<span class="token punctuation">.</span>missingMethodArguments<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">var</span> propName <span class="token operator">=</span> <span class="token function">right</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>missingMethodName<span class="token punctuation">,</span> <span class="token function">len</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>missingMethodName<span class="token punctuation">)</span> <span class="token operator">-</span> <span class="token number">3</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">var</span> newValue <span class="token operator">=</span> missingMethodArguments<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token comment">//we know there's only 1 arg from the check above</span><br /> <span class="token keyword">return</span> <span class="token function">__set__</span><span class="token punctuation">(</span> propName<span class="token punctuation">,</span> <span class="token function">isNull</span><span class="token punctuation">(</span>newValue<span class="token punctuation">)</span> <span class="token operator">?</span> <span class="token function">nullvalue</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">:</span> newValue <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">invoke</span><span class="token punctuation">(</span>variables<span class="token punctuation">.</span>entity<span class="token punctuation">,</span> missingMethodName<span class="token punctuation">,</span> missingMethodArguments<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">__set__</span><span class="token punctuation">(</span> <span class="token parameter">required string propName<span class="token punctuation">,</span> newValue</span> <span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">var</span> <span class="token function-variable function">getter</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token function">invoke</span><span class="token punctuation">(</span>variables<span class="token punctuation">.</span>entity<span class="token punctuation">,</span> <span class="token string">'get'</span> <span class="token operator">&</span> propName<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /> <span class="token keyword">var</span> <span class="token function-variable function">setter</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span> <span class="token parameter">x</span> <span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token function">invoke</span><span class="token punctuation">(</span>variables<span class="token punctuation">.</span>entity<span class="token punctuation">,</span> <span class="token string">'set'</span> <span class="token operator">&</span> propName<span class="token punctuation">,</span> <span class="token punctuation">[</span><span class="token function">isNull</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span> <span class="token operator">?</span> <span class="token function">nullvalue</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">:</span> x<span class="token punctuation">]</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">isNull</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>newValue<span class="token punctuation">)</span> <span class="token operator">&&</span> <span class="token function">isNull</span><span class="token punctuation">(</span><span class="token function">getter</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token comment">//don't track this "change"</span><br /> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">__isTrackableProp__</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>propName<span class="token punctuation">,</span> arguments<span class="token punctuation">.</span>newValue<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">var</span> currentVal <span class="token operator">=</span> <span class="token function">getter</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><br /> <span class="token punctuation">(</span><span class="token function">isNull</span><span class="token punctuation">(</span><span class="token function">getter</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token operator">&&</span> <span class="token operator">!</span><span class="token function">isNull</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>newValue<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token operator">||</span> <span class="token punctuation">(</span><span class="token operator">!</span><span class="token function">isNull</span><span class="token punctuation">(</span><span class="token function">getter</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token operator">&&</span> <span class="token function">isNull</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>newValue<span class="token punctuation">)</span><span class="token punctuation">)</span><br /> <span class="token operator">||</span> <span class="token punctuation">(</span>currentVal <span class="token operator">!=</span> arguments<span class="token punctuation">.</span>newValue<span class="token punctuation">)</span><br /> <span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> variables<span class="token punctuation">.</span>changes<span class="token punctuation">.</span><span class="token function">append</span><span class="token punctuation">(</span><span class="token punctuation">{</span><br /> <span class="token string-property property">'field'</span><span class="token operator">:</span> <span class="token function">lcase</span><span class="token punctuation">(</span><span class="token function">left</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>propName<span class="token punctuation">,</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token operator">&</span> <span class="token function">right</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>propName<span class="token punctuation">,</span> <span class="token function">len</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>propName<span class="token punctuation">)</span> <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token string-property property">'from'</span><span class="token operator">:</span> <span class="token function">isNull</span><span class="token punctuation">(</span>currentVal<span class="token punctuation">)</span> <span class="token operator">?</span> <span class="token string">'(NULL)'</span> <span class="token operator">:</span> currentVal<span class="token punctuation">,</span><br /> <span class="token string-property property">'to'</span><span class="token operator">:</span> <span class="token function">isNull</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>newValue<span class="token punctuation">)</span> <span class="token operator">?</span> <span class="token string">'(NULL)'</span> <span class="token operator">:</span> arguments<span class="token punctuation">.</span>newValue<br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token function">setter</span><span class="token punctuation">(</span><span class="token function">isNull</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>newValue<span class="token punctuation">)</span> <span class="token operator">?</span> <span class="token function">nullvalue</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">:</span> arguments<span class="token punctuation">.</span>newValue<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">__isTrackableProp__</span><span class="token punctuation">(</span><span class="token parameter">required string propName<span class="token punctuation">,</span> newValue</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//if you're trying to SET an entity as a value, we can't track that...</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span><span class="token function">isNull</span><span class="token punctuation">(</span>newValue<span class="token punctuation">)</span> <span class="token operator">&&</span> <span class="token function">isObject</span><span class="token punctuation">(</span>newValue<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token comment">//if it's an array, that can only be useful to pass an array of entities for a relationship</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span><span class="token function">isNull</span><span class="token punctuation">(</span>newValue<span class="token punctuation">)</span> <span class="token operator">&&</span> <span class="token function">isArray</span><span class="token punctuation">(</span>newValue<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token comment">//if the current value is an entity, we can't track that...</span><br /> <span class="token keyword">var</span> current <span class="token operator">=</span> <span class="token function">invoke</span><span class="token punctuation">(</span>variables<span class="token punctuation">.</span>entity<span class="token punctuation">,</span> <span class="token string">'get'</span> <span class="token operator">&</span> propName<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span><span class="token function">isNull</span><span class="token punctuation">(</span>current<span class="token punctuation">)</span> <span class="token operator">&&</span> <span class="token function">isObject</span><span class="token punctuation">(</span>current<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token comment">//if it's an array, that can only be useful to pass an array of entities for a relationship</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span><span class="token function">isNull</span><span class="token punctuation">(</span>current<span class="token punctuation">)</span> <span class="token operator">&&</span> <span class="token function">isArray</span><span class="token punctuation">(</span>current<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token comment">//otherwise, we should be able to track that...</span><br /> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">public</span> <span class="token keyword">function</span> <span class="token function">__getEntity__</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> variables<span class="token punctuation">.</span>entity<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">public</span> <span class="token keyword">function</span> <span class="token function">__getChanges__</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> variables<span class="token punctuation">.</span>changes<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
Choose Your Hard Isn't Cancelled2023-10-12T00:00:00Zhttps://adamtuttle.codes/blog/2023/choose-your-hard-isnt-cancelled/<p class="photo-byline"><img src="https://adamtuttle.codes/img/2023/towfiqu-barbhuiya-Yw9Vgr6i_-0-unsplash.jpg" alt="An old fashioned metal key" />
Photo by <a href="https://unsplash.com/@towfiqu999999?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Towfiqu barbhuiya</a> on <a href="https://unsplash.com/photos/Yw9Vgr6i_-0?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a></p>
<p>Every now and then the <a href="https://duckduckgo.com/?va=n&t=hv&q=choose+your+hard&ia=web">"choose your hard"</a> meme spikes on social media. Some are quick to denounce it as a false choice because not everything hard is a choice. They're missing the point.</p>
<p>Choose your hard is about <em>recognizing which hard things are choices</em>. Of course you didn't choose to get cancer, or to have a child die, or any of myriad horrible things. But you do get to choose how you respond to those things. You can choose to be bitter, or you can choose to be better. <strong>You can choose to be a victim, or you can choose to be a survivor.</strong></p>
<p>I have a pretty debilitating auto-immune disease that, untreated, would "turn my spine into bamboo" to use the metaphor that my doctor used to explain it to me. Now that I'm getting older, my vision is starting to go. I've had better-than-20/20 vision for my entire life, and now that I have even just the slightest astigmatism (multiple optometrists questioned whether I actually needed glasses), 5 minutes of looking at a computer screen without my glasses gives me a headache. I have moderate tinnitus that is exacerbated by the white-noise machine that my wife can't sleep without. Both of my children have been hospitalized for significant amounts of time, requiring me to miss work and stay with them (not to mention the stress of having children with that level of healthcare needs).</p>
<p>I could go on for 5 more paragraphs.</p>
<p>None of these things were choices. All of them "happened to me" and they are all hard on their own.</p>
<p>I choose to take the medication that makes living in this body bearable. For the first 10ish years that was an I.V. infusion every 6 weeks. Then it was at-home injections. Now it's a daily pill. Next year it could be something else.</p>
<p>I choose to wear glasses, and take care of them.</p>
<p>I choose to sleep with earphones in so that I can listen to old movies while I drift off rather than slowly go insane from the ringing in my ears.</p>
<p>I choose to be grateful for the time that I have with my children, and to be grateful for the medical professionals who have helped them. I choose to be grateful that I am a parent in 2023, and not 1823.</p>
<h3 id="but-it's-more-than-that" tabindex="-1">But it's more than that</h3>
<p>Not only do I choose to look at the bright side of the hard things that have happened to me, I choose which hard things I'm going to do while not letting my hardships hold me back.</p>
<p><strong>Body wants to turn to stone?</strong> In 2014 I got my skydiving license. I am a parachute rigger. I have 2 different skydiving instructional ratings and I have nearly 800 jumps. (By the way, if you're ever in south-eastern PA and want to make a jump, hit me up!)</p>
<p>I choose not only to make my living by making computers do my bidding, but to constantly follow my ambition to keep getting better at it. I choose to keep learning new things, and to keep teaching others what I've learned.</p>
<p>I chose to write a book because it sounded like a fun challenge.</p>
<p>I make fine furniture<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2023/choose-your-hard-isnt-cancelled/#fn1" id="fnref1">[1]</a></sup> from local hardwoods. I turn bowls on my lathe. I find joy in making things, and I choose not to be discouraged by the projects that end up in the burn pit.</p>
<h3 id="choose-what-motivates-you" tabindex="-1">Choose what motivates you</h3>
<p>"Choose your hard" spikes on social media because it's pithy, and it works for a lot of people.</p>
<p>It reminds me that while working up the motivation to exercise can be hard today, being overweight and out of shape is hard but on a different time scale. I don't know yet if my body is capable of getting the phsyique I want, but I know that I can choose to work towards it regardless.</p>
<p>Nuance is important in this discussion, too. If "choose your hard" doesn't work for you, that's ok. But that doesn't mean it can't work for anyone. Likewise, I'm sure that some people take the choose your hard thing too far and come off as judgemental of you and your choices. That's not ok, and I'm sorry if that's been your experience.</p>
<p>The idea of "not yucking someone else's yum" works both ways.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>Well, I try. I'm not half bad. But I'm no master, either. Lots of work to do in that department. 😅 <a href="https://adamtuttle.codes/blog/2023/choose-your-hard-isnt-cancelled/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
What to Do When You’re Drowning in Tech Debt2023-03-03T00:00:00Zhttps://adamtuttle.codes/blog/2023/what-to-do-when-drowning-in-technical-debt/<p class="photo-byline"><img src="https://adamtuttle.codes/img/2023/hunters-race-MYbhN8KaaEc-unsplash.jpg" alt="An old fashioned metal key" />
Photo by <a href="https://unsplash.com/@huntersrace?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Hunters Race</a> on <a href="https://unsplash.com/photos/MYbhN8KaaEc?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<h2 id="where-does-it-come-from%3F" tabindex="-1">Where does it come from?</h2>
<p>It's not <em>dark science</em> or <em>rocket surgery</em> to write decent software. You can get lucky and slap something together that works on the first try and never has problems, but you can't count on that luck to last your entire career. The good news is that there's a consistent and predictable way to write good software.</p>
<ol>
<li>Decide what feature needs to be written. If complexity necessitates, write a specification.</li>
<li>Write tests that will validate when you've reached the goal. When the tests pass, stop working. If your tests are passing and you know there's more features to add or more edge cases to handle, then you need to add more failing tests first.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2023/what-to-do-when-drowning-in-technical-debt/#fn1" id="fnref1">[1]</a></sup></li>
<li>Write the code that makes the tests pass, ignoring best practices – get to passing tests as quick and dirty as you can!</li>
<li>Refactor the code to follow all of those best practices you ignored in the last step. Fortunately you now have the tests to give you confidence that you haven't broken anything during your refactoring.
<ul>
<li>Remove duplication</li>
<li>Employ <a href="https://en.wikipedia.org/wiki/SOLID">SOLID principles</a></li>
<li>Make it readable for the developer who has to debug it in 10 years and doesn't know what you're thinking today. It might not be you.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2023/what-to-do-when-drowning-in-technical-debt/#fn2" id="fnref2">[2]</a></sup></li>
<li>etc...</li>
</ul>
</li>
<li>Submit for code review, and incorporate feedback as appropriate.</li>
</ol>
<h3 id="then%2C-why-is-there-always-so-much-technical-debt%3F" tabindex="-1">Then, why is there always so much technical debt?</h3>
<p><strong>I believe that we allow "the business" to get in the way of doing our jobs well.</strong></p>
<p>Too often, software companies and IT departments are given a project mandate and an allowable time limit to spend working on it. Maybe the CEO talked to customers about it, decided it would be a valuable project, and the customer would need it by a certain date in order to be able to use it.</p>
<p>That's the wrong way to think about it.</p>
<p>Even if we are diligent about only working on the parts that <em>really must be included</em> and not getting sidetracked by things that are interesting, the phrase "it'll be done when it's done" is still right; regardless of whether the business wants it done in 3 months or 3 weeks.</p>
<p>So you've accepted a project, and an unrealistic timeline. Now what happens?</p>
<p>Well, you can't skip step 3 (<em>write the code</em>), because writing the code is the whole point. And step 5 (<em>code review</em>) seems like it could be a good safeguard against the problems that steps 1 (<em>specification</em>) and 2 (<em>tests</em>) protect you from, and code review tends to take a lot less time than writing a spec and tests. And step 4 (<em>refactoring</em>)? You think the business is going to let you take code that's already working make it "better" without making any functional changes? That's going to go right out the window.</p>
<p>What's left?</p>
<ol>
<li>Write code</li>
<li>Get a code review</li>
</ol>
<p>Sound familiar? I bet it does.</p>
<p><strong>Skipping those steps is where technical debt is born.</strong></p>
<h3 id="professional-responsibility" tabindex="-1">Professional Responsibility</h3>
<p>What if you had to sign a document stating that your code is engineered correctly and safe to go into production, and that you may be held personally responsible if it breaks and causes financial, reputational, or other realizable harm to the business? This is not that far off from the responsibility that architects and construction site engineers bear.</p>
<p>Would an architect allow her design to move on to construction without confidence that the specified girders can hold the weight of the building and all of the people and equipment it will one day need to hold? Not one who wants to keep her license.</p>
<p>Would a construction site engineer allow the building to start going up before the foundation is complete? Not one who wants to keep her job.</p>
<p>These aren't perfect metaphors, probably mostly because I don't know the first thing about architecture or construction. But my point is that these jobs are more than jobs, they're professions.</p>
<p>You're not the guy who puts the sour cream on the tacos when they land in front of you, you're a professional, dammit. Act like one.</p>
<p style="text-align:center">* * *</p>
<p>Your job isn't to translate the human-readable specification into computer-readable code. It's to be a software engineer. When an architect or an engineer tells their customer that the request can't be completed safely in the given time, the customer will accept that and find other ways to make their building schedule work, or they'll have to find another architect or engineer.</p>
<p>The Archictects and Site Engineers <em>are</em> responsible, and they <em>will</em> be held accountable if the building crumbles.</p>
<p>Everyone acknowledges safety is important. Shouldn't it be important in business software, too?</p>
<p>If you know you're not being given enough time to complete the project well, refuse to sign the paperwork that says that you can be held responsible for failure, because you know it inevitably will fail.</p>
<p>Of course, we're not expected to sign documents like that. So what does the metaphorical signature-refusal look like in the real world?</p>
<p>Explain to the business that good work takes time and you can't arbitrarily decide it needs to be done faster. You can't skip the tests and expect robust software, just like you can't pour dry concrete mix into a hole and hope that the rain will soak the concrete and cure before its strength is needed. If you want a reliable product, there are steps you can't skip.</p>
<p>Do they want a hack thrown together by a team of under-slept coffee addicts whose nerves are shot from the on-call alarm buzzing constantly, or do they want the right solution, engineered properly to support the business in its activities and for the concept that there's software making the business possible (dare I say easy) to be invisible?</p>
<p style="text-align:center">* * *</p>
<p>I am of course championing the concept of <a href="https://37signals.com/seven-shipping-principles#appetite">Appetite</a> in software planning.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2023/what-to-do-when-drowning-in-technical-debt/#fn3" id="fnref3">[3]</a></sup></p>
<p>The short version is that if the timeline must be fixed, then the scope must be flexible. If there's not enough time to do the software engineering right, and to get it all done, then features have to start getting dropped. Start with the most important features, and be ruthless about cutting things.</p>
<p>You can have a fixed scope, or a fixed timeline, but not both.</p>
<h3 id="step-1%3A-stop-the-bleeding" tabindex="-1">Step 1: Stop the bleeding</h3>
<p>You'll never escape from technical debt if you don't slow down its creation.</p>
<p>An ounce of prevention is worth a pound of cure. If we as an industry start standing up for ourselves and enforce the right way of doing our jobs, then future technical debt can be minimized if not avoided entirely. And we can get some sleep, too!</p>
<p>Truth be told, I don't think most businesses will have an appreciation for technical debt until they're being told it's what's causing their developers to move more slowly than they expected. Sometimes you have to suffer a little bit before you're willing to invest in the process. Which brings us nicely to existing tech debt.</p>
<h3 id="what-about-pre-existing-tech-debt%3F" tabindex="-1">What about pre-existing tech debt?</h3>
<p>I am a big fan of <a href="https://stackoverflow.blog/2023/02/27/stop-saying-technical-debt/">framing technical debt as "maintenance load."</a> It helps the business understand the true cost of technical debt: If your maintenance load is 50% then a 6 week project will take 12 weeks to complete. It also gives us, the engineers, a concrete objective to battle against. Identifying specific sources of maintenance load and reducing or eliminating them will (<strong>measurably!</strong>) improve the lives of the whole team, allow the department to ship projects faster, and help the business do... business things... better.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2023/what-to-do-when-drowning-in-technical-debt/#fn4" id="fnref4">[4]</a></sup></p>
<p>It could be a difficult conversation, but if you frame it as something concrete, and you bring actual metrics to the table to show the sources and cost of your current maintenance load, then you can have a real discussion about what can be done to reduce it and prevent it going forward.</p>
<p>The business doesn't want to give you 3 weeks to make the software "better" without adding any new features, but spending 3 weeks to invest in shipping future projects faster, when framed in maintenance load terms, can be seen as a win even in their non-technical eyes.</p>
<h3 id="estimating%3A-the-footgun-of-software-engineering" tabindex="-1">Estimating: The Footgun of Software Engineering</h3>
<p>I acknowledge that not every project is brought down from the mountain with a requirements doc and an unrealistic deadline pre-attached. Sometimes we estimate ourselves into this situation, and in those cases we have only ourselves to blame. I don't have any silver bullets (yet?) for doing better at estimating, but here are some thoughts to get you started.</p>
<p>I've heard people say that they come up with their estimation, and then they double or triple it, and give that as the actual estimation, to cover anything they haven't thought of. That's one way to do it, though probably imprecise.</p>
<p>I think it's more likely that we just forget to think about planning, testing, refactoring, and code review —and edge-cases!— when we make estimations, because writing code quickly can be a point of pride. <em>You need 4 weeks to write that? I could do it in 2!</em> But never in the history of ever has <em>I could do it in 2!</em> guy been thinking about planning, testing, refactoring, edge-cases, and code review when he gives that number.</p>
<p>Don't be that guy.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>It is a legitimate strategy to write a prototype of the feature first before you write tests if you don't yet understand what it will take to implement the feature. You can't test something you don't understand, so write a throw-away prototype to learn from, then throw it away, and write your tests for the real implementation. <a href="https://adamtuttle.codes/blog/2023/what-to-do-when-drowning-in-technical-debt/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>"Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. Code for readability." - <a href="https://stackoverflow.com/a/878436/751">John Woods</a> <a href="https://adamtuttle.codes/blog/2023/what-to-do-when-drowning-in-technical-debt/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p>I am not a fan of DHH / 37Signals, but this was the first place I became aware of the concept. Credit where it's due, I guess. <a href="https://adamtuttle.codes/blog/2023/what-to-do-when-drowning-in-technical-debt/#fnref3" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn4" class="footnote-item"><p>I am so good at wording. <a href="https://adamtuttle.codes/blog/2023/what-to-do-when-drowning-in-technical-debt/#fnref4" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Encrypting AWS RDS and EBS After Deploy2023-02-28T00:00:00Zhttps://adamtuttle.codes/blog/2023/encrypting-aws-rds-and-ebs-after-deploy/<p class="photo-byline"><img src="https://adamtuttle.codes/img/2023/everyday-basics-GJY1eAw6tn8-unsplash.jpg" alt="An old fashioned metal key" />
Photo by <a href="https://unsplash.com/@zanardi?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Everyday basics</a> on <a href="https://unsplash.com/photos/GJY1eAw6tn8?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>We unexpectedly found ourselves with some AWS RDS databases using unencrypted storage; as well as some EC2 instances using root EBS volumes that were unencrypted.</p>
<p><em>Crap.</em></p>
<p>For a variety of reasons, that's not good.</p>
<p>And look, I'm not here to point fingers, but some chowder-head (me! 👉 😩 👈) put us into this situation, and now we need to get ourselves out of it. So, how do you do that with minimal downtime?</p>
<p>I'll take EBS first, because that one is simpler, and I did it first.</p>
<h3 id="encrypting-the-root-ebs-volume-of-an-ec2-instance" tabindex="-1">Encrypting the root EBS volume of an EC2 instance</h3>
<p>I believe that AWS will allow you to encrypt-in-place —while the instance is running, even!— but not if it's the root volume.</p>
<p>Here's what I ended up doing:</p>
<ol>
<li>If you're regularly generating files and storing them on the volume, and you can't lose any of them during the brief downtime, then you'll need to shut down the instance first and be prepared for a few extra minutes of down time. On the other hand, if files are rarely written and you don't mind if logs are missing a couple of minutes of history, you can take a snapshot while the instance is running and only shut the instance down for a minute or two when we get to step 4 below. Either way, you need to make a snapshot of the volume.</li>
<li>Make an encrypted copy of the snapshot. Choose the make a copy action, and there's a checkbox to encrypt the copy. Make sure that your snapshot copy is in the same region as the existing volume. The copy wizard doesn't pre-fill smartly.</li>
<li>Once you have your encrypted snapshot (copy), create a new encrypted volume from your encrypted snapshot.</li>
<li>You're now ready to shut down the instance (if you haven't already) and swap the volumes.
<ol>
<li>Before you detach the unencrypted volume, note the device name. (e.g. <code>/dev/hda1</code>) You must attach the new volume with the same device name.</li>
<li>Detach the unencrypted volume</li>
<li>Attach the encrypted volume (with the same device name)</li>
</ol>
</li>
<li>Restart your EC2 instance.</li>
</ol>
<p>I did about a half dozen of these one Saturday morning, and got them all done in less than an hour.</p>
<h3 id="encrypting-the-storage-of-an-rds-instance" tabindex="-1">Encrypting the storage of an RDS instance</h3>
<p>This one's trickier. If you're anything like us, your databases use a lot more storage than your app servers. The approach is going to be mostly the same, but of course you probably can't (we definitely couldn't) allow the data to change between snapshot and encrypted restore. So, with hat in hand, I informed our affected customers that we would be having a scheduled maintenance window on a Sunday afternoon for about 5 hours. (Spoiler alert: I finished with about 30 minutes to spare!)</p>
<p>I don't know if it will affect the available options, but it's worth mentioning: We're using AWS Aurora (MySQL Compatible).</p>
<ol>
<li>Block all web traffic to the applications that use the database. I added an ALB rule that returned a fixed 503 response with a "down for scheduled maintenance" page.</li>
<li>Disable all scheduled microservice executions that would affect the database. For us, this meant shutting down some Fargate services, and disabling some EventBridge events that would trigger Lambdas on a schedule.</li>
<li>Snapshot the database. <strong>DO NOT</strong> "stop" the database. You can't snapshot a database while it's stopped, so it's pointless at this step. It also takes a <em>long time</em> to stop a database, and even longer to start it back up.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2023/encrypting-aws-rds-and-ebs-after-deploy/#fn1" id="fnref1">[1]</a></sup></li>
<li>Now that you have your database snapshot, <strong>DO</strong> "stop" the database. That's the best way to prevent any unexpected DB changes. Fortunately you don't have to wait for the stop to complete before you can move on to the next step.</li>
<li>Restore from your snapshot. This will take you to a wizard to create a new cluster. Make sure that you choose to encrypt the new cluster!</li>
<li>Once you've submitted the cluster creation wizard, creation will begin, and it will be immediately assigned a hostname. You probably need to put the updated hostname into some configs somewhere, right? Now's the time to do that, and do any necessary deploys to get the new hostname out to all the places that need to use it.</li>
<li>You can add additional instances to the cluster before the first one is complete, if you want.</li>
<li>Once the cluster created and fully started and available, connect to it with your DBMS software and poke around to make sure it looks as expected. Got all of the databases and users? Data looks recent?</li>
<li>Allow traffic to once again flow to the web apps and microservices. Don't forget to re-enable any schedules that you disabled.</li>
</ol>
<p>I spent about 2 hours doing the database work described above, and another 2.5 or so updating configs and doing deploys to get the updated hostname config everywhere it needed to be. I made some mistakes along the way that slowed me down, but thankfully I estimated conservatively and managed to get it all done within the allotted maintenance window.</p>
<h3 id="encrypted-by-default" tabindex="-1">Encrypted By Default</h3>
<p>If you're running a business, you should be using encrypted storage. Not only does it just make sense, but sooner or later you're going to run into compliance regulations that will require it, so you might as well choose it from the start.</p>
<p>You can also enable encryption by default to help you and your team choose the right option when creating new EBS volumes. Here's <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html#encryption-by-default">how to enable encryption by default for EBS volumes</a>. I can't find a document on enabling encryption by default for RDS, but I bet it's possible somehow.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>Ask anyone who's accidentally stopped the wrong database. (Speaking from experience...) <a href="https://adamtuttle.codes/blog/2023/encrypting-aws-rds-and-ebs-after-deploy/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Fix Missing CSS when Using Tailwind to Write a Svelte Component Library2023-02-13T00:00:00Zhttps://adamtuttle.codes/blog/2023/tailwind-svelte-design-system/<p class="photo-byline"><img src="https://adamtuttle.codes/img/2023/deva-darshan-Jt9syHEhrPE-unsplash.jpg" alt="Cloverleaf shaped highway interchange" />
Photo by <a href="https://unsplash.com/@darshan394?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Deva Darshan</a> on <a href="https://unsplash.com/photos/Jt9syHEhrPE?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>In my free time (because I am a giant nerd) I have been working on a Svelte-based design system for my company to use. I'm using Tailwind to style my components, because Tailwind is awesome. (<em>Fight me.</em>)</p>
<p>To make it maximally reusable, it is its own project which we'll install using npm.</p>
<p>Unfortunately, while I got pretty far building components, when I went to implement them in an application, I found that the Tailwind styling I applied in the design system project wasn't being (completely) adhered to in the parent application.</p>
<p>The class names were still present in the generated HTML, but the Tailwind JIT compiler wasn't detecting that they were being used, and including those in the generated CSS. Any classes that were also used in the parent app would be in the generated CSS, and they would work on my components, but anything unique to the components would be missing. I am <a href="https://github.com/svelte-add/svelte-add/issues/180">not alone in running into this problem</a>.</p>
<p>I had started down the path of writing a custom solution to use a Svelte action that would inspect the <code>class</code> attribute of the element and convert the class names into the equivalent CSS rules and add them to the <code>style</code> attribute...</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>div</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>bg-slate-300 hover:bg-slate-100<span class="token punctuation">"</span></span> <span class="token attr-name"><span class="token namespace">use:</span>tailwindToCSS</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>div</span><span class="token punctuation">></span></span></code></pre>
<p>While this was a fun distraction to think about for a little while, it's misguided. It can't handle hover states, responsive design, and other dynamic features like dark mode -- at least not without adding a bunch of JS, which was something I didn't really want to do.</p>
<p>Fortunately, there are a lot of really smart people on the Tailwind Discord server, and one of them put me on the right path. (Thanks, <a href="https://github.com/crswll">Bill Criswell</a>!)</p>
<p>Using Tailwind in Svelte requires the use of a <code>tailwind.config.cjs</code> file, which looks like this by default:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> config <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">content</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">'./src/**/*.{html,js,svelte,ts}'</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">theme</span><span class="token operator">:</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">extend</span><span class="token operator">:</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">plugins</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br />module<span class="token punctuation">.</span>exports <span class="token operator">=</span> config<span class="token punctuation">;</span></code></pre>
<p>If our design system is available inside of <code>node_modules/...</code> then all we need to do is tell Tailwind to parse those files, too...</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> config <span class="token operator">=</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">content</span><span class="token operator">:</span> <span class="token punctuation">[</span></span><br /><span class="highlight-line"> <span class="token string">'./src/**/*.{html,js,svelte,ts}'</span><span class="token punctuation">,</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token string">'./node_modules/@COMPANY/DESIGN_SYSTEM/**/*.{html,js,svelte,ts}'</span></mark><br /><span class="highlight-line"> <span class="token punctuation">]</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">theme</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">extend</span><span class="token operator">:</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">plugins</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">]</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line">module<span class="token punctuation">.</span>exports <span class="token operator">=</span> config<span class="token punctuation">;</span></span></code></pre>
<p>While this probably isn't an ideal situation if you want to create a public design system (ala Bootstrap), because requiring people to setup Tailwind and modify its config seems like it might be asking a bit much, it's certainly a fine solution for something like a company-internal design system, and it's the approach I intend to use.</p>
How to Subscribe to Changes of Individual Properties Within a Svelte Store2022-12-09T00:00:00Zhttps://adamtuttle.codes/blog/2022/how-to-subscribe-to-svelte-store-sub-property-changes/<p>Svelte stores are great. You can easily subscribe to them to be notified when a value changes, and react accordingly.</p>
<pre class="language-ts"><code class="language-ts"><span class="token comment">//store.ts</span><br /><span class="token keyword">import</span> <span class="token punctuation">{</span> writable <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'svelte/store'</span><span class="token punctuation">;</span><br /><span class="token keyword">import</span> <span class="token keyword">type</span> <span class="token punctuation">{</span> Filter <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'./ListFilterTypes'</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">export</span> <span class="token keyword">const</span> emptyFilter<span class="token operator">:</span> Filter <span class="token operator">=</span> <span class="token punctuation">{</span><br /> filterId<span class="token operator">:</span> <span class="token keyword">null</span><span class="token punctuation">,</span><br /> filterName<span class="token operator">:</span> <span class="token string">''</span><span class="token punctuation">,</span><br /> operator<span class="token operator">:</span> <span class="token keyword">null</span><span class="token punctuation">,</span><br /> not<span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span><br /> value<span class="token operator">:</span> <span class="token string">''</span><span class="token punctuation">,</span><br /> min<span class="token operator">:</span> <span class="token string">''</span><span class="token punctuation">,</span><br /> max<span class="token operator">:</span> <span class="token string">''</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">export</span> <span class="token keyword">const</span> filter <span class="token operator">=</span> <span class="token generic-function"><span class="token function">writable</span><span class="token generic class-name"><span class="token operator"><</span>Filter<span class="token operator">></span></span></span><span class="token punctuation">(</span>Object<span class="token punctuation">.</span><span class="token function">assign</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> emptyFilter<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<pre class="language-js"><code class="language-js"><span class="token comment">//App.svelte</span><br /><span class="token keyword">import</span> <span class="token punctuation">{</span> filter <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'./store'</span><span class="token punctuation">;</span><br /><br />filter<span class="token punctuation">.</span><span class="token function">subscribe</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">$filter</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token comment">/* do awesome stuff */</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>But stores like the one above, which contain an entire object with various properties in it, always notify subscribers if <em>any</em> property changes. What if you have a special case where you want a subscription that only executes when the <code>value</code> property is updated?</p>
<p>I scratched my head on this for a few minutes. At first I thought there might be an extra argument to the <code>subscribe</code> method that I could make use of. A quick scan of the docs showed that there wasn't. So what's one to do?</p>
<h2 id="use-a-derived-store" tabindex="-1">Use a derived store</h2>
<p>Derived stores allow you to create a new store that is based on the value of an existing store. Any time the first store is updated the derived store gets updated.</p>
<p>So we can easily add a derived store to our <code>store.ts</code> file:</p>
<pre class="language-ts"><code class="language-ts"><mark class="highlight-line highlight-line-active"><span class="token keyword">import</span> <span class="token punctuation">{</span> writable<span class="token punctuation">,</span> derived <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'svelte/store'</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">export</span> <span class="token keyword">const</span> filter <span class="token operator">=</span> <span class="token generic-function"><span class="token function">writable</span><span class="token generic class-name"><span class="token operator"><</span>Filter<span class="token operator">></span></span></span><span class="token punctuation">(</span>Object<span class="token punctuation">.</span><span class="token function">assign</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> emptyFilter<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><mark class="highlight-line highlight-line-active"><span class="token keyword">export</span> <span class="token keyword">const</span> filterValue <span class="token operator">=</span> <span class="token function">derived</span><span class="token punctuation">(</span>filter<span class="token punctuation">,</span> <span class="token punctuation">(</span>$filter<span class="token punctuation">)</span> <span class="token operator">=></span> $filter<span class="token punctuation">.</span>value<span class="token punctuation">)</span><span class="token punctuation">;</span></mark></code></pre>
<p>Now there's a store named <code>filterValue</code> that will be updated only when <code>filter.value</code> changes. In all other ways we can treat it like a normal store, which means that subscribing to <code>filter.value</code> changes is super easy:</p>
<pre class="language-js"><code class="language-js"><span class="token comment">//App.svelte</span><br /><span class="token keyword">import</span> <span class="token punctuation">{</span> filter<span class="token punctuation">,</span> filterValue <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'./store'</span><span class="token punctuation">;</span><br /><br />filterValue<span class="token punctuation">.</span><span class="token function">subscribe</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">$filterValue</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token comment">/* do awesome stuff that only needs to happen when filter.value changes */</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
Lessons Learned From Migrating Large MySQL Databases2022-11-14T00:00:00Zhttps://adamtuttle.codes/blog/2022/lessons-learned-migrating-large-mysql-databases/<p class="photo-byline"><img src="https://adamtuttle.codes/img/2022/chris-briggs-V72Hk6LjjjI-unsplash.jpg" alt="Sandhill Cranes against a cloudy sky" />
Photo by <a href="https://unsplash.com/@cgbriggs19?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Chris Briggs</a> on <a href="https://unsplash.com/s/photos/migration?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>My team and I migrated 13 customer databases from MySQL 5.x to 8.x last week. It was hell.</p>
<p>Most crimes of database inefficiency can fly under the radar if your database is small enough.</p>
<p>My databases weren't small enough.</p>
<p>For reference, our largest tables were > 100gb but < 150gb.</p>
<p>I work for a startup, on a small team where every team member wears every hat. We're all the DBA and the Architect and the Code Monkey and the Project Manager and the Helpdesk and the DevOps and the Site Reliability Engineers. I'm sure that most or all of what I'm about to describe is already well known by people who specialize in databases, or even people in similar roles to me who have more experience.</p>
<p>Sometimes you have to learn the hard way by fighting the process until 3am and figuring it out from first principles and lots of web searches.</p>
<h2 id="lesson-%231%3A-import-data-with-constraints-disabled" tabindex="-1">Lesson #1: Import data with constraints disabled</h2>
<p>All of these lessons require careful thought, but this one especially so.</p>
<pre class="language-sql"><code class="language-sql"><span class="token keyword">set</span> foreign_key_checks <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span><br /><span class="token comment">/* import data here */</span><span class="token punctuation">;</span><br /><span class="token keyword">set</span> foreign_key_checks <span class="token operator">=</span> <span class="token number">1</span></code></pre>
<p>This allows you to import the data in whatever order makes the most sense to you, even if it violates referential integrity, without losing any of it. It's also faster because it doesn't have to check foreign keys for every row being inserted.</p>
<p><em>You're disabling guard rails that keep you safe, so you have to take extra care to know exactly what you're doing</em>, but this can be really helpful, because of the speed boost.</p>
<p>In some cases we saw a 6gb file import take as little as 7 minutes.</p>
<h2 id="lesson-%232%3A-add-multiple-indexes-with-a-single-alter" tabindex="-1">Lesson #2: Add multiple indexes with a single alter</h2>
<p><strong>Naive:</strong></p>
<pre class="language-sql"><code class="language-sql"><span class="token keyword">create</span> <span class="token keyword">index</span> ixFoo <span class="token keyword">using</span> <span class="token keyword">btree</span> <span class="token keyword">on</span> Fubar<span class="token punctuation">(</span>foo<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token keyword">create</span> <span class="token keyword">index</span> ixBar <span class="token keyword">using</span> <span class="token keyword">btree</span> <span class="token keyword">on</span> Fubar<span class="token punctuation">(</span>bar<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token keyword">create</span> <span class="token keyword">index</span> ixBaz <span class="token keyword">using</span> <span class="token keyword">btree</span> <span class="token keyword">on</span> Fubar<span class="token punctuation">(</span>baz<span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p><strong>Better:</strong></p>
<pre class="language-sql"><code class="language-sql"><span class="token keyword">alter</span> <span class="token keyword">table</span> Fubar<br /> <span class="token keyword">add</span> <span class="token keyword">key</span> <span class="token string">'ixFoo'</span> <span class="token punctuation">(</span><span class="token string">'foo'</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token keyword">add</span> <span class="token keyword">key</span> <span class="token string">'ixBar'</span> <span class="token punctuation">(</span><span class="token string">'bar'</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token keyword">add</span> <span class="token keyword">key</span> <span class="token string">'ixBaz'</span> <span class="token punctuation">(</span><span class="token string">'baz'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p><strong>Why it's better:</strong></p>
<p>When you add an index to a table, the entire table is copied to a temp table, your alteration is made, and then the temp table is hot-swapped in to replace the original.</p>
<pre class="language-sql"><code class="language-sql"><span class="token comment">/* how a table hot swap works, hypothetically */</span><br /><span class="token keyword">begin</span> <span class="token keyword">transaction</span><span class="token punctuation">;</span><br /><span class="token keyword">drop</span> <span class="token keyword">table</span> Fubar<span class="token punctuation">;</span><br /><span class="token keyword">rename</span> <span class="token keyword">table</span> tmpFubar <span class="token keyword">to</span> Fubar<span class="token punctuation">;</span><br /><span class="token keyword">commit</span><span class="token punctuation">;</span></code></pre>
<p>Now imagine your table is hundreds of gigabytes, with a bunch of indexes. In the naive approach, for each index being added, the table has to be copied, altered, and renamed. For the better approach, the table is copied only once.</p>
<h2 id="lesson-%233%3A-split-the-work-up-to-use-the-best-tool-for-the-data-you-need-to-move" tabindex="-1">Lesson #3: Split the work up to use the best tool for the data you need to move</h2>
<p><code>mysqldump</code> is great, but not the best approach when you've got a bunch of tables that are hundreds of gigabytes each that need to be moved. Since we're on AWS, and our MySQL databases are actually <a href="https://aws.amazon.com/rds/aurora/">Aurora MySQL</a>, we were able to use some special syntax to <a href="https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.SaveIntoS3.html">export data to CSV's on S3</a> and then <a href="https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.LoadFromS3.html">load it back into MySQL</a> on the new instance.</p>
<p>We also heavily used <code>mysqldump</code> to handle the migration of smaller tables. That's why this lesson is labeled "split the work up;" because sometimes it makes sense to use <code>mysqldump</code> and sometimes it doesn't. Anything under 1gb we especially didn't even consider worth ... considering. But for those really big tables, here's how we exported them to S3 and then into the new database server:</p>
<pre class="language-sql"><code class="language-sql"><span class="token keyword">select</span> <span class="token operator">*</span> <span class="token keyword">from</span> Fubar <span class="token keyword">into</span> <span class="token keyword">outfile</span> s3 <span class="token string">'s3-us-east-1://bucket-name/Fubar.csv'</span><br /><span class="token keyword">fields</span> <span class="token keyword">terminated</span> <span class="token keyword">by</span> <span class="token string">','</span><br /><span class="token keyword">optionally</span> <span class="token keyword">enclosed</span> <span class="token keyword">by</span> <span class="token string">'"'</span><br /><span class="token keyword">escaped</span> <span class="token keyword">by</span> <span class="token string">"\\"</span><br /><span class="token keyword">lines</span> <span class="token keyword">terminated</span> <span class="token keyword">by</span> <span class="token string">'\n'</span><br />overwrite <span class="token keyword">on</span></code></pre>
<p>This generates CSV files, and automatically splits them at 6GB increments, which makes for easy transfer between servers, and easy import using <code>load data from s3</code>:</p>
<pre class="language-sql"><code class="language-sql"><span class="token keyword">load</span> <span class="token keyword">data</span> <span class="token keyword">from</span> s3 <span class="token string">'s3-us-east-1://bucket/fubar.csv.part_00000'</span><br /><span class="token keyword">replace</span> <span class="token keyword">into</span> <span class="token keyword">table</span> Fubar<br /><span class="token keyword">fields</span> <span class="token keyword">terminated</span> <span class="token keyword">by</span> <span class="token string">','</span><br /><span class="token keyword">optionally</span> <span class="token keyword">enclosed</span> <span class="token keyword">by</span> <span class="token string">'"'</span><br /><span class="token keyword">escaped</span> <span class="token keyword">by</span> <span class="token string">'\\'</span><br /><span class="token keyword">lines</span> <span class="token keyword">terminated</span> <span class="token keyword">by</span> <span class="token string">'\n'</span><br /><span class="token punctuation">(</span><br /> <span class="token variable">@col1</span><br /> <span class="token punctuation">,</span> <span class="token variable">@col2</span><br /><span class="token punctuation">)</span><br /><span class="token keyword">set</span><br /> col1 <span class="token operator">=</span> <span class="token keyword">nullif</span><span class="token punctuation">(</span><span class="token variable">@col1</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><br /> <span class="token punctuation">,</span> col2 <span class="token operator">=</span> <span class="token keyword">nullif</span><span class="token punctuation">(</span><span class="token variable">@col2</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><br /><span class="token punctuation">;</span></code></pre>
<p><em>Repeat the <code>load data</code> step for each 6gb chunk, e.g. <code>.part_00001</code>, <code>.part_00002</code>, etc.</em></p>
<p>The approach described above is great for moving really large tables to a new db server instance. You wouldn't want to do this for all tables, unless they're all really large, because it adds a bunch of manual steps. When you do it this way, you have to create the tables manually and you'll want to wait until the data is inserted before adding the indexes so that the indexes don't have to be updated for every record you insert.</p>
<p>But this approach is better than using <code>mysqldump</code> for large tables specifically because it allows you to import the data before the indexes are added. I bet there are arguments to <code>mysqldump</code> to make it not create indexes or to affect when it adds them, but by default it includes them in the <code>create table</code> statements.</p>
<p>This approach also gives you the opportunity to:</p>
<ol>
<li>import only a subset of the data (in our case, the most recent data)</li>
<li>add the indexes</li>
<li>bring the application back online so that users can continue working, and then</li>
<li>allow the rest of the data to restore over time, albeit more slowly because of the constant index rebuilding.</li>
</ol>
<p>This is what we ended up doing.</p>
<p>After waiting for <em>more than 6 hours</em> for 5 indexes to be added to a single table after importing its entire history, we stumbled our way into the idea of importing only the most recent data for each table, then adding the indexes. This allowed our app to run in a slightly degraded state, but it was online. Then we queued up the rest of the imports (<code>load data from s3</code>) and went to bed. When I woke up in the morning, I checked in on it and the data loads were complete.</p>
<p>This approach also requires some special care.</p>
<p>Our tables use <code>auto_increment</code> primary keys. When we created the tables, we knew what the maximum PK values were, and we added a generous buffer to the <code>auto_increment</code> start value that would be used for newly inserted records. Is this strictly necessary? Probably not. But it was a healthy conservative approach.</p>
<p>If the <code>fubar</code> table had a maximum PK value of <code>155,834,091</code> then we set the <code>auto_increment</code> to <code>200,000,000</code></p>
<pre class="language-sql"><code class="language-sql"><span class="token keyword">create</span> <span class="token keyword">table</span> <span class="token identifier"><span class="token punctuation">`</span>fubar<span class="token punctuation">`</span></span> <span class="token punctuation">(</span><br /> <span class="token comment">/* ... */</span><br /><span class="token punctuation">)</span> <span class="token keyword">engine</span><span class="token operator">=</span><span class="token keyword">innodb</span> <span class="token keyword">auto_increment</span><span class="token operator">=</span><span class="token number">200000000</span><span class="token punctuation">;</span></code></pre>
<p>This ensures that all new inserts that happen while the application is running have PK values that won't conflict with rows that we still need to import.</p>
How to Use TypeScript with Svelte and Vite in 20222022-11-07T00:00:00Zhttps://adamtuttle.codes/blog/2022/svelte-and-ts-2022/<p>As of the time of this writing, Svelte uses Vite for builds, and while Vite will automatically handle <code>.ts</code> files, it won't handle TypeScript in <code>.svelte</code> files.</p>
<p>For this to work, you need to preprocess your <code>.svelte</code> files using <a href="https://github.com/sveltejs/svelte-preprocess">svelte-preprocess</a>. Unfortunately their guides seem to be a bit outdated (references to Rollup abound). So I figured it out for myself (not that it was that hard) and I'm sharing it here for you and for future me.</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">npm</span> i <span class="token parameter variable">-D</span> svelte-preprocess</code></pre>
<p>Once you have <code>svelte-preprocess</code> installed, create a <code>svelte.config.js</code> in the root of your svelte project (e.g. same folder as your <code>package.json</code>) and paste this content into it:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">import</span> preprocess <span class="token keyword">from</span> <span class="token string">'svelte-preprocess'</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">const</span> config <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">preprocess</span><span class="token operator">:</span> <span class="token function">preprocess</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">export</span> <span class="token keyword">default</span> config<span class="token punctuation">;</span></code></pre>
<p>That's it. Vite builds will now preprocess your <code>.svelte</code> files, and <code>svelte-preprocess</code> has baked-in support for TypeScript, so builds should now stop throwing these errors because the Svelte compiler doesn't recognize your TypeScript:</p>
<blockquote>
<p>Error [ParseError]: Unexpected token</p>
</blockquote>
The Working Code Test2022-08-03T00:00:00Zhttps://adamtuttle.codes/blog/2022/the-working-code-test/<!-- Photo by <a href="https://unsplash.com/@joshuanewton?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Joshua Newton</a> on <a href="https://unsplash.com/s/photos/test?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a> -->
<p>In today's episode of the <a href="https://workingcode.dev/">Working Code Podcast</a>, I discuss the infamous <strong><a href="https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-steps-to-better-code/">Joel Test</a></strong> for grading a software team with my cohost <a href="https://www.bennadel.com/">Ben Nadel</a>.</p>
<p>We dug deep on what's the same – sometimes astonishingly so! – 22 years later, and what should be reconsidered or revised. We acknowledged that at the time Joel was probably primarily focused on producing a product that was a binary executable that could be put onto a CD-ROM and sold on a store shelf; and that the SaaS space has exploded (in a good way) since then. So we evaluated the list through the lens of a web-development, SaaS product engineer.</p>
<p>It was the year 2000 when he wrote the original. Just months after I graduated from high school. So it is with tremendous respect and appreciation for the work of those that came before me that I try to do it justice by giving it the glow-up it deserves.</p>
<iframe allow="autoplay *; encrypted-media *; fullscreen *; clipboard-write" frameborder="0" height="175" style="width:100%;max-width:660px;overflow:hidden;background:transparent;" sandbox="allow-forms allow-popups allow-same-origin allow-scripts allow-storage-access-by-user-activation allow-top-navigation-by-user-activation" src="https://embed.podcasts.apple.com/us/podcast/086-the-working-code-test/id1544142288?i=1000574879560"></iframe>
<p>While we covered the items in order, it may have been a little difficult to follow, so here then is our revised list:</p>
<ol>
<li>Do you use source control?</li>
<li>Can you deploy to production in a single step?</li>
<li>Do you use continuous integration?</li>
<li>Do you have a bug database?</li>
<li>Do you fix bugs before writing new code? (where appropriate)</li>
<li>Do you have a roadmap?</li>
<li>Do programmers have quiet working conditions?</li>
<li>Do you use the best tools money can buy?</li>
<li>Do you require extensive automated tests for critical path and mission critical features?</li>
<li>Do new candidates write code during their interview?</li>
<li>Do you do hallway usability testing?</li>
<li>Is the existing team diverse and does it value the benefits of diversity?</li>
<li>Are engineers given the freedom to participate in ideation and the freedom to figure out the best solution, rather than implementing a rigid specification?</li>
<li>Do you include accessibility concerns in testing and code reviews?</li>
<li>Do engineers get face-to-face contact with customers and the support team?</li>
<li>Do you automate almost everything that can be automated?</li>
<li>Do you send your people to training and conferences?</li>
<li>Do you set career growth goals for your people?</li>
</ol>
Global node.js Variables Are Safe to Use in AWS Lambda2022-07-26T00:00:00Zhttps://adamtuttle.codes/blog/2022/aws-lambda-safe-global-variables/<p><img src="https://adamtuttle.codes/img/2022/sigmund-aI4RJ--Mw4I-unsplash.jpg" alt="Close up photograph of the "recycle" symbol on a recycling bin" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@sigmund?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Sigmund</a> on <a href="https://unsplash.com/s/photos/recycle?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>In general, global variables are <a href="https://stackoverflow.com/questions/5063878/how-to-cleanly-deal-with-global-variables">frowned upon</a>.</p>
<p>In fact, they can be dangerous. If you have a Node.js application running an Express web server, and you read and write global variables at various points during a request, it is a near certainty that you'll experience race conditions and variable leakage under even modest request concurrency.</p>
<p>Given the multiple-request model of Express<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2022/aws-lambda-safe-global-variables/#fn1" id="fnref1">[1]</a></sup>, if you write a global variable during a request, and then later reference that same global variable during the same request, there is no guarantee that it will contain the same value that you previously wrote.</p>
<p><strong>So then, it is <em>incredibly important</em> to understand where and how your application is running, and what the benefits and limitations of that environment are.</strong></p>
<p>The way that Amazon Web Services implements <a href="https://aws.amazon.com/blogs/compute/understanding-aws-lambda-scaling-and-throughput/">concurrency for Lambda functions</a> is, to over-simplify, that each function can have up to N concurrently-available containers ("environments" if you don't want to use a word that evokes Docker), but that each container will have its own segregated memory, and each container will only ever handle 1 request at a time. Once a request completes, its container can be reused if there are pending requests, and any global state left at the end of the first request will be set and available at the start of the next.</p>
<p>That's it. That's the point. If you're good enough at reading between the lines, that's all there is to learn here.</p>
<h2 id="but-why-would-you-do-this%3F" tabindex="-1">But why would you do this?</h2>
<p>As previously mentioned, global variables are generally frowned upon. So why is it a good thing that they're safe to use in this scenario? Here's a concrete example that illustrates the benefit.</p>
<p>Suppose you're writing a Lambda function and you want to log every exception that it throws to some external logging service (other than just the default CloudWatch logs). First, you write a utility function to send the thrown exceptions to your logging service:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">logError</span><span class="token punctuation">(</span><span class="token parameter">error<span class="token punctuation">,</span> context</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> logResult <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">fetch</span><span class="token punctuation">(</span><span class="token string">'https://...'</span><span class="token punctuation">,</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">method</span><span class="token operator">:</span> <span class="token string">'POST'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">body</span><span class="token operator">:</span> <span class="token constant">JSON</span><span class="token punctuation">.</span><span class="token function">stringify</span><span class="token punctuation">(</span><span class="token punctuation">{</span> error<span class="token punctuation">,</span> context <span class="token punctuation">}</span><span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>logResult<span class="token punctuation">.</span>ok<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> console<span class="token punctuation">.</span><span class="token function">error</span><span class="token punctuation">(</span><span class="token string">'Error while sending an exception to the log service! 😱'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> console<span class="token punctuation">.</span><span class="token function">error</span><span class="token punctuation">(</span><span class="token punctuation">{</span> error<span class="token punctuation">,</span> context <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>This function makes an HTTP POST request containing the error object and the context object serialized to JSON for logging purposes. The <code>error</code> argument is self-explanatory: it's the error that was thrown. Perhaps less obvious is <code>context</code>. The intention here is to give the developer the opportunity to provide any additional details that might be helpful in diagnosing the problem. For example, what were the inputs to the Lambda request? What were some midpoint findings during the execution, before the exception was thrown? Having these details can be the difference between finding a root cause and not being able to diagnose an issue at all.</p>
<p>Now we need to use it in our Lambda function:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> logError <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'./logError'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br />exports<span class="token punctuation">.</span><span class="token function-variable function">handler</span> <span class="token operator">=</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token parameter">event</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> s3Bucket <span class="token operator">=</span> event<span class="token punctuation">.</span>Records<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>s3<span class="token punctuation">.</span>bucket<span class="token punctuation">.</span>name<span class="token punctuation">;</span><br /> <span class="token keyword">let</span> found <span class="token operator">=</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token keyword">let</span> count <span class="token operator">=</span> <span class="token number">42</span><span class="token punctuation">;</span><br /> <span class="token keyword">try</span> <span class="token punctuation">{</span><br /> <span class="token comment">//do something useful</span><br /> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span>error<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">logError</span><span class="token punctuation">(</span>error<span class="token punctuation">,</span> <span class="token punctuation">{</span> found<span class="token punctuation">,</span> count<span class="token punctuation">,</span> s3Bucket <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p>Again, this seems pretty straight-forward, and global variable usage hasn't shown itself to be useful yet.</p>
<p>But.</p>
<p>Are your Lambda functions truly that simple? An average one of mine might have 5 or 6 local utility libraries for managing database and redis access, piping messages to our team chat, validation, and so on. There's not just layers to the functionality, there are <em>strata</em>.</p>
<p>Consider that you might want to collect a bunch of information along the journey through your code to include in an error log, if the need should arise. You could pass it down through each method call. Equally tedious, if you want to access some context from several layers up the stack for an error thrown much lower.</p>
<p>Here I'm highlighting all of the lines that are less clean than they could be, because of the prop drilling.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">exports<span class="token punctuation">.</span><span class="token function-variable function">handler</span> <span class="token operator">=</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token parameter">event</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">const</span> s3Bucket <span class="token operator">=</span> event<span class="token punctuation">.</span>Records<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>s3<span class="token punctuation">.</span>bucket<span class="token punctuation">.</span>name<span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token keyword">let</span> found <span class="token operator">=</span> <span class="token boolean">false</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token keyword">let</span> count <span class="token operator">=</span> <span class="token number">42</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token comment">//... useful things ...</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">await</span> <span class="token function">makeWidgets</span><span class="token punctuation">(</span>s3Bucket<span class="token punctuation">,</span> <span class="token punctuation">{</span> found<span class="token punctuation">,</span> count <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><mark class="highlight-line highlight-line-active"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">makeWidgets</span><span class="token punctuation">(</span><span class="token parameter">bucket<span class="token punctuation">,</span> context</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></mark><br /><span class="highlight-line"> <span class="token comment">//... useful things ...</span></span><br /><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>inventoryCount <span class="token operator"><</span> minInventoryCount<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">await</span> <span class="token function">warnAboutInventoryCount</span><span class="token punctuation">(</span>bucket<span class="token punctuation">,</span> context<span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span></span><br /><span class="highlight-line"></span><br /><mark class="highlight-line highlight-line-active"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">warnAboutInventoryCount</span><span class="token punctuation">(</span><span class="token parameter">bucket<span class="token punctuation">,</span> context</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></mark><br /><span class="highlight-line"> <span class="token comment">//what if something goes wrong?</span></span><br /><span class="highlight-line"> <span class="token keyword">try</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token comment">//... useful things ...</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span>error<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token function">logError</span><span class="token punctuation">(</span>error<span class="token punctuation">,</span> context<span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>For those familiar with React.js, this feels a lot like the <a href="https://kentcdodds.com/blog/prop-drilling">prop drilling</a><sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2022/aws-lambda-safe-global-variables/#fn2" id="fnref2">[2]</a></sup> problem. Potentially many functions in the middle of a chain of function calls need to support context as input and pass it on to functions called later, without ever using it or adding to it, all because something at the end of the chain might need it. This gets tedious and feels <em>bad</em> and <em>wrong</em>, even if you can't articulate why.</p>
<h2 id="global-variables-to-the-rescue" tabindex="-1">Global Variables to the Rescue</h2>
<p class="call-out">It should be noted that this is not a general purpose solution that will work in all scenarios, but in the specific scope of this article (AWS Lambda), this is a useful tool.</p>
<p>If we make a small tweak to our error logging function to expect and use a global context variable, and add a method for appending additional context for any later error reports, then our code becomes cleaner and easier to understand and maintain, and accomplishes the same goal.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">addDebugContext</span><span class="token punctuation">(</span><span class="token parameter">context</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> global<span class="token punctuation">.</span>debugContext <span class="token operator">=</span> Object<span class="token punctuation">.</span><span class="token function">assign</span><span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> global<span class="token punctuation">.</span>debugContext<span class="token punctuation">,</span> context<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">logError</span><span class="token punctuation">(</span><span class="token parameter">error<span class="token punctuation">,</span> context</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>context<span class="token punctuation">)</span> <span class="token function">addDebugContext</span><span class="token punctuation">(</span>context<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">const</span> logResult <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">fetch</span><span class="token punctuation">(</span><span class="token string">'https://...'</span><span class="token punctuation">,</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">method</span><span class="token operator">:</span> <span class="token string">'POST'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">body</span><span class="token operator">:</span> <span class="token constant">JSON</span><span class="token punctuation">.</span><span class="token function">stringify</span><span class="token punctuation">(</span><span class="token punctuation">{</span> error<span class="token punctuation">,</span> <span class="token literal-property property">context</span><span class="token operator">:</span> global<span class="token punctuation">.</span>debugContext <span class="token punctuation">}</span><span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>logResult<span class="token punctuation">.</span>ok<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> console<span class="token punctuation">.</span><span class="token function">error</span><span class="token punctuation">(</span><span class="token string">'Error while sending an exception to the log service! 😱'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> console<span class="token punctuation">.</span><span class="token function">error</span><span class="token punctuation">(</span><span class="token punctuation">{</span> error<span class="token punctuation">,</span> <span class="token literal-property property">context</span><span class="token operator">:</span> global<span class="token punctuation">.</span>debugContext <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>With the updated logging context approach, our application can be simplified thusly:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> <span class="token punctuation">{</span> addDebugContext<span class="token punctuation">,</span> logError <span class="token punctuation">}</span> <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'./logError'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br />exports<span class="token punctuation">.</span><span class="token function-variable function">handler</span> <span class="token operator">=</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token parameter">event</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> s3Bucket <span class="token operator">=</span> event<span class="token punctuation">.</span>Records<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>s3<span class="token punctuation">.</span>bucket<span class="token punctuation">.</span>name<span class="token punctuation">;</span><br /> <span class="token keyword">let</span> found <span class="token operator">=</span> <span class="token boolean">false</span><span class="token punctuation">;</span><br /> <span class="token keyword">let</span> count <span class="token operator">=</span> <span class="token number">42</span><span class="token punctuation">;</span><br /> <span class="token function">addDebugContext</span><span class="token punctuation">(</span><span class="token punctuation">{</span> found<span class="token punctuation">,</span> count <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token comment">//... useful things ...</span><br /> <span class="token keyword">await</span> <span class="token function">makeWidgets</span><span class="token punctuation">(</span>s3Bucket<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">makeWidgets</span><span class="token punctuation">(</span><span class="token parameter">bucket</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token comment">//... useful things ...</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>inventoryCount <span class="token operator"><</span> minInventoryCount<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">await</span> <span class="token function">warnAboutInventoryCount</span><span class="token punctuation">(</span>bucket<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">warnAboutInventoryCount</span><span class="token punctuation">(</span><span class="token parameter">bucket</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token comment">//what if something goes wrong?</span><br /> <span class="token keyword">try</span> <span class="token punctuation">{</span><br /> <span class="token comment">//... useful things ...</span><br /> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span>error<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">logError</span><span class="token punctuation">(</span>error<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>With this approach, the job of storing useful context for debugging is done along side the creation or discovery of that context, and doesn't need to be whispered down the lane in order to be available for inclusion in any incidental error logging that happens anywhere down that rabbit hole.</p>
<p>Of course, this leaves the previous invocation's context in place when the next invocation starts, so you'll want to do a "reset" at the start of every invocation just to make sure the global variables are in a clean and useful state for the invocation that's starting:</p>
<pre class="language-js"><code class="language-js"><span class="token comment">//add this to logError.js</span><br /><span class="token keyword">function</span> <span class="token function">resetDebugContext</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> global<span class="token punctuation">.</span>debugContext <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> <span class="token punctuation">{</span> resetDebugContext<span class="token punctuation">,</span> addDebugContext<span class="token punctuation">,</span> logError <span class="token punctuation">}</span> <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'./logError'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line">exports<span class="token punctuation">.</span><span class="token function-variable function">handler</span> <span class="token operator">=</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token parameter">event</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token function">resetDebugContext</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//clear context from previous invocations</span></mark><br /><span class="highlight-line"> <span class="token comment">// ...</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>I don't mean to pick on Express here, it's not a problem unique to their implementation, but it's an ecosystem that the average Node.js developer should be familiar with, so it's a useful model for illustration purposes. <a href="https://adamtuttle.codes/blog/2022/aws-lambda-safe-global-variables/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>The irony is not lost on me: I'm advocating for usage of global variables (<em>in a well-defined scenario!</em>) and linking to an article that says that prop drilling is a useful solution for the problem of using global variables. I agree that globals are bad in a React application for all of the reasons Kent explains; but I contend that this different situation deserves separate consideration. <a href="https://adamtuttle.codes/blog/2022/aws-lambda-safe-global-variables/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Debugging Docker Production Builds2022-06-29T00:00:00Zhttps://adamtuttle.codes/blog/2022/debugging-docker-production-builds/<p><img src="https://adamtuttle.codes/img/2021/barrett-ward-5WQJ_ejZ7y8-unsplash.jpg" alt="Shipping containers in a busy port" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@barrettward?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Barrett Ward</a> on <a href="https://unsplash.com/s/photos/container?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>Filing this one under things that I'm writing for my future self to reference.</p>
<p>Who among us hasn't had the experience where you're getting a bug report for a bug that you know you fixed, and it seems like your fix is just being ignored in production? (If you haven't had that experience, I envy you!)</p>
<p>This is especially frustrating when your production environment is a Docker container running in the cloud, built by a CI/CD server. You can't exactly SSH into the server and look at the code to make sure your deploy <em>actually worked</em>. I guess you could unblock that, but it seems like the wrong reason to open up SSH ports on your production containers and probably inadvisable.</p>
<p>Anyway, if you find yourself in this situation, here's how you can double-check what's actually inside that production container. My notes are from Amazon's AWS Elastic Container Repository (ECR) but the steps should be the same even if you're using a different service and the syntax varies.</p>
<ol>
<li>Log in to ECR:</li>
</ol>
<pre class="language-bash"><code class="language-bash">aws ecr get-login-password <span class="token parameter variable">--region</span> us-east-1 <span class="token operator">|</span> <span class="token function">docker</span> login <span class="token parameter variable">--username</span> AWS --password-stdin <span class="token operator"><</span>account-id<span class="token operator">></span>.dkr.ecr.us-east-1.amazonaws.com</code></pre>
<ol start="2">
<li>Pull the image from ECR to your local machine:</li>
</ol>
<pre class="language-bash"><code class="language-bash"><span class="token function">docker</span> pull <span class="token operator"><</span>account-id<span class="token operator">></span>.dkr.ecr.us-east-1.amazonaws.com/my/repo/name:tagname</code></pre>
<ol start="3">
<li>Start the container locally:</li>
</ol>
<pre class="language-bash"><code class="language-bash"><span class="token function">docker</span> run <span class="token parameter variable">-it</span> <span class="token parameter variable">--entrypoint</span> /bin/bash <span class="token operator"><</span>account-id<span class="token operator">></span>.dkr.ecr.us-east-1.amazonaws.com/my/repo/name:tagname</code></pre>
<p>This starts the container in interactive mode and rather than launching its usual startup command, runs, in this case, <code>/bin/bash</code> to give you a shell to browse around the container.</p>
<p>Now you can dig around and make sure your code changes are in fact represented in the production build, where and how you expected.</p>
<p>What should you do if the code is what you wanted it to be and the app still doesn't work?</p>
<p>Well, friend... That remains one of life's great mysteries.</p>
Scaling Fargate Based On SQS Queue Depth2022-06-13T00:00:00Zhttps://adamtuttle.codes/blog/2022/scaling-fargate-based-on-sqs-queue-depth/<p>Something I've talked about a fair amount on the <a href="https://workingcode.dev/">Working Code Podcast</a> recently has been a project where I needed to take some work that was being initiated by API requests and run in the foreground, and move those off-server and ideally queue them up so that we never have to worry about two tasks fighting over resources or, worse, deadlocking. I initially planned to follow <a href="https://www.blog.muraliallada.com/fargate-scaling/">the approach laid out by Murali Allada</a>, and in some ways I did, but what we ended up doing reduced the number of steps and made it feel less like a rube-goldberg machine.</p>
<p>His approach involved using a CloudWatch alarm to trigger an SNS message, that would start a Lambda function, which would call the AWS API to scale your Fargate containers as needed. It presumably works for him, but the number of hand-offs does make me worry that it's a bit fragile.</p>
<p>My approach<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2022/scaling-fargate-based-on-sqs-queue-depth/#fn1" id="fnref1">[1]</a></sup> gets rid of the SNS message and the Lambda function, and directly scales Fargate from the CloudWatch alarm. This may not have been possible back in June of 2020 when Murali wrote his article – which goes to show how AWS is always changing and improving.</p>
<p><img src="https://adamtuttle.codes/img/2022/queue-scaling.png" alt="A flow chart showing SQS flowing into CloudWatch Alarms, which flows into SNS, which flows into Lambda, which flows into Fargate; but SNS and Lambda have been scratched out with a big squiggly line." /></p>
<h2 id="not-all-rainbows-and-lollipops" tabindex="-1">Not all rainbows and lollipops</h2>
<p>One thing that I'm not particularly happy about with this configuration is that it sometimes takes 15+ minutes after a long period of queue dormancy for CloudWatch to notice that there are items in the queue. If you're a regular user of CloudWatch, some lag will not be a surprise to you. I believe that in their docs they promise a max of something like 15 minutes from the time the item is added to the queue until the time that CloudWatch is aware of it. While not ideal, this is <em>acceptable</em> for our needs. Remember that this only applies because our queue will lie dormant for most of the day, so it seems like CloudWatch stops paying close attention to it. If your queue will be in more or less constant usage, this might not be a problem for you.</p>
<p>An alternative approach if you would experience this lag but don't want to wait 15+ minutes for queue work to start processing would be to have SQS directly trigger a Lambda, and in your Lambda do the inspections and math to determine if you need to scale up or scale down. With this approach you're taking control over something that AWS can do for you, so you get extra control but you're also responsible for keeping it working.</p>
<h2 id="how-to-create-the-infrastructure" tabindex="-1">How to create the infrastructure</h2>
<p>The first thing you need is an SQS queue that you want to monitor, and a Fargate service that you want to scale based on your queue's depth (item count).</p>
<p>From there, you need to create at least two CloudWatch Alarms. I started by creating them manually in the AWS console using <a href="https://www.lastweekinaws.com/blog/clickops/">click-ops</a>. Depending on what you expect your typical queue usage to be you might need to alter the values I'll provide here, but for my project I expect the queue to be empty most of the time, and then in the early morning a bunch of tasks will be pushed into it. Once the work is complete, the queue will be going dormant again until the next morning.</p>
<p>So I need an alarm that detects we've transitioned from zero to one-or-more items in the queue so that I can scale my Fargate instance from 0 desired instances to 1; and I need another alarm to detect that the work is done and scale Fargate back down to 0 desired instances.</p>
<p>The first one is pretty straight forward.</p>
<ul>
<li>Select your queue</li>
<li>Monitor the metric <code>ApproximateNumberOfMessagesVisible</code></li>
<li>Statistic: <code>minimum</code></li>
<li>Period: 1 minute</li>
<li>Threshold
<ul>
<li>Type: Static</li>
<li>Condition: <code>Greater/Equal</code></li>
<li>Value: <code>1</code></li>
</ul>
</li>
<li>Datapoints to alarm: <code>1</code> out of <code>1</code></li>
<li>Treat missing data as missing</li>
</ul>
<p>This alarm will trigger the moment it sees anything appear in the queue. We'll circle back to hooking it up to Fargate later.</p>
<p>Next we need an alarm that will determine that work is done and it can scale Fargate back down to 0 desired instances. We can't simply rely on there being 0 items in the SQS queue, because each task from the queue takes a certain amount of time to complete. If your alarm scales down your Fargate worker before it's able to get the work done, then the job will never complete. I'm not sure which is worse: if the failure to complete causes the item to reappear in the queue and start the process over again, only to be shutdown prematurely every time, eventually landing in your Dead Letter Queue; or if you were to delete the item from the queue immediately and then get shutdown prematurely then the job would never complete and never be retried. Either way is pretty bad.</p>
<p>So what you need to decide is how much time needs to pass from the moment that the task is received from the SQS Queue (leaving the queue now empty) until you can be sure that the job has completed. For argument's sake, let's say 10 minutes. So you need to create an alarm that triggers after the queue has been empty for 10 minutes.</p>
<ul>
<li>Select your queue</li>
<li>Monitor the metric <code>ApproximateNumberOfMessagesVisible</code></li>
<li>Statistic: <code>maximum</code></li>
<li>Period: 1 minute</li>
<li>Threshold
<ul>
<li>Type: Static</li>
<li>Condition: <code>Lower/Equal</code></li>
<li>Value: <code>0</code></li>
</ul>
</li>
<li>Datapoints to alarm: <code>10</code> out of <code>10</code></li>
<li>Treat missing data as missing</li>
</ul>
<p>With this alarm, if a new item appears in the queue after 8 minutes then the countdown will have to restart after that new queue item has been removed.</p>
<p>From here, you can start hooking your alarms up to your Fargate services. You can select them in the auto-scaling section of the service wizard (again with the click-ops).</p>
<p>This works great as long as you don't need to publish a new version of your task runner. For some strange reason, when I uploaded a new container to ECR and deployed it for my service, the alarm would be removed. Not very helpful!</p>
<p>So this forced my hand to step back from click-ops, which is probably a good thing, and to build some automation around these deploys. I'm going to consider the ECR and ECS changes out-of-scope for this (already quite long) article, and only show you the additional steps I had to take to reconnect the alarms and Fargate services.</p>
<p>For <a href="https://www.merriam-webster.com/dictionary/belt-and-suspenders">belt-and-suspenders</a> reasons I decided to recreate the Alarms, Scaling Policies, and Scalable Targets for every deploy. The all overwrite if existing, so there's no harm and it simplifies everything a bit.</p>
<p>If you know me, you know that I am a big fan of Makefiles, so here's the commands from my Makefile:</p>
<ol>
<li>First, the scalable target:</li>
</ol>
<pre class="language-bash"><code class="language-bash">@aws application-autoscaling register-scalable-target --service-namespace ecs --scalable-dimension ecs:service:DesiredCount --resource-id service/YOUR-CLUSTER-NAME/YOUR-SERVICE-NAME --min-capacity <span class="token number">0</span> --max-capacity <span class="token number">1</span></code></pre>
<ol start="2">
<li>Then, the scaling policies:</li>
</ol>
<pre class="language-bash"><code class="language-bash">@aws application-autoscaling put-scaling-policy --service-namespace ecs --scalable-dimension ecs:service:DesiredCount --resource-id service/YOUR-CLUSTER-NAME/YOUR-SERVICE-NAME --policy-name scale-out --policy-type StepScaling --step-scaling-policy-configuration file://<span class="token variable"><span class="token variable">`</span><span class="token builtin class-name">pwd</span><span class="token variable">`</span></span>/aws/scaling-policies/scale-out.json <span class="token operator">></span> /dev/null<br /><br />@aws application-autoscaling put-scaling-policy --service-namespace ecs --scalable-dimension ecs:service:DesiredCount --resource-id service/YOUR-CLUSTER-NAME/YOUR-SERVICE-NAME --policy-name scale-in --policy-type StepScaling --step-scaling-policy-configuration file://<span class="token variable"><span class="token variable">`</span><span class="token builtin class-name">pwd</span><span class="token variable">`</span></span>/aws/scaling-policies/scale-in.json <span class="token operator">></span> /dev/null</code></pre>
<p>The policies use input files that are located relative to the Makefile at <code>aws/scaling-policies/...</code>. Here's an example of the <code>scale-out.json</code> file:</p>
<pre class="language-json"><code class="language-json"><span class="token punctuation">{</span><br /> <span class="token property">"AdjustmentType"</span><span class="token operator">:</span> <span class="token string">"ExactCapacity"</span><span class="token punctuation">,</span><br /> <span class="token property">"Cooldown"</span><span class="token operator">:</span> <span class="token number">300</span><span class="token punctuation">,</span><br /> <span class="token property">"MetricAggregationType"</span><span class="token operator">:</span> <span class="token string">"Minimum"</span><span class="token punctuation">,</span><br /> <span class="token property">"StepAdjustments"</span><span class="token operator">:</span> <span class="token punctuation">[</span><br /> <span class="token punctuation">{</span><br /> <span class="token property">"MetricIntervalLowerBound"</span><span class="token operator">:</span> <span class="token number">0.0</span><span class="token punctuation">,</span><br /> <span class="token property">"MetricIntervalUpperBound"</span><span class="token operator">:</span> <span class="token number">5.0</span><span class="token punctuation">,</span><br /> <span class="token property">"ScalingAdjustment"</span><span class="token operator">:</span> <span class="token number">1</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">{</span><br /> <span class="token property">"MetricIntervalLowerBound"</span><span class="token operator">:</span> <span class="token number">5.0</span><span class="token punctuation">,</span><br /> <span class="token property">"ScalingAdjustment"</span><span class="token operator">:</span> <span class="token number">2</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">]</span><br /><span class="token punctuation">}</span></code></pre>
<p>This file is more complex than the scaling steps I described above. It will scale to 1 instance from 0 to 5 items in the queue, but if more than 5 items are in the queue it will scale up to 2 desired instances of the service.</p>
<ol start="3">
<li>Next we need to (re)create the alarms, and tell them to use the ARNs of the scaling policies we just created.</li>
</ol>
<pre class="language-bash"><code class="language-bash">@aws cloudwatch put-metric-alarm --alarm-actions $<span class="token variable"><span class="token variable">$(</span>aws application-autoscaling describe-scaling-policies --service-namespace ecs <span class="token parameter variable">--query</span> <span class="token string">'ScalingPolicies[?PolicyName==`scale-out`].PolicyARN'</span> <span class="token parameter variable">--output</span> text<span class="token variable">)</span></span> --cli-input-json file://<span class="token variable"><span class="token variable">`</span><span class="token builtin class-name">pwd</span><span class="token variable">`</span></span>/aws/alarms/queue.json<br /><br />@aws cloudwatch put-metric-alarm --alarm-actions $<span class="token variable"><span class="token variable">$(</span>aws application-autoscaling describe-scaling-policies --service-namespace ecs <span class="token parameter variable">--query</span> <span class="token string">'ScalingPolicies[?PolicyName==`scale-in`].PolicyARN'</span> <span class="token parameter variable">--output</span> text<span class="token variable">)</span></span> --cli-input-json file://<span class="token variable"><span class="token variable">`</span><span class="token builtin class-name">pwd</span><span class="token variable">`</span></span>/aws/alarms/dequeue.json</code></pre>
<p>Here again, we're also passing in some JSON files to specify most of the attributes of our alarms. Here's a sample of <code>queue.json</code>:</p>
<pre class="language-json"><code class="language-json"><span class="token punctuation">{</span><br /> <span class="token property">"AlarmName"</span><span class="token operator">:</span> <span class="token string">"warehouse-queue"</span><span class="token punctuation">,</span><br /> <span class="token property">"ActionsEnabled"</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> <span class="token property">"OKActions"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token property">"InsufficientDataActions"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token property">"MetricName"</span><span class="token operator">:</span> <span class="token string">"ApproximateNumberOfMessagesVisible"</span><span class="token punctuation">,</span><br /> <span class="token property">"Namespace"</span><span class="token operator">:</span> <span class="token string">"AWS/SQS"</span><span class="token punctuation">,</span><br /> <span class="token property">"Statistic"</span><span class="token operator">:</span> <span class="token string">"Minimum"</span><span class="token punctuation">,</span><br /> <span class="token property">"Dimensions"</span><span class="token operator">:</span> <span class="token punctuation">[</span><br /> <span class="token punctuation">{</span><br /> <span class="token property">"Name"</span><span class="token operator">:</span> <span class="token string">"QueueName"</span><span class="token punctuation">,</span><br /> <span class="token property">"Value"</span><span class="token operator">:</span> <span class="token string">"warehouse-queue"</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token property">"Period"</span><span class="token operator">:</span> <span class="token number">60</span><span class="token punctuation">,</span><br /> <span class="token property">"EvaluationPeriods"</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span><br /> <span class="token property">"DatapointsToAlarm"</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span><br /> <span class="token property">"Threshold"</span><span class="token operator">:</span> <span class="token number">1.0</span><span class="token punctuation">,</span><br /> <span class="token property">"ComparisonOperator"</span><span class="token operator">:</span> <span class="token string">"GreaterThanOrEqualToThreshold"</span><span class="token punctuation">,</span><br /> <span class="token property">"TreatMissingData"</span><span class="token operator">:</span> <span class="token string">"missing"</span><br /><span class="token punctuation">}</span></code></pre>
<p>We run these 3 updates (scalable target, scaling policy, alarm) after every deploy of the worker containers. They only add a few seconds, and it guarantees that everything is still wired correctly. More importantly, it doesn't require a human to click around in the console and hope they got it right.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>Hat tip to my teammate Crump for the idea! <a href="https://adamtuttle.codes/blog/2022/scaling-fargate-based-on-sqs-queue-depth/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Semaphore 0.3 and 1.02022-05-10T00:00:00Zhttps://adamtuttle.codes/blog/2022/semaphore-0.3-and-1.0/<p><a href="https://adamtuttle.codes/blog/2021/introducing-semaphore/">Semaphore</a> is an open source library I wrote that helps add feature flags to CFML projects.</p>
<p>Today I am releasing Semaphore <a href="https://github.com/atuttle/semaphore/releases/tag/v0.3.0">version 0.3</a>, which includes no functional changes, but serves as a ⚠️ <strong>deprecation warning</strong> ⚠️ for breaking changes coming in v1.0, soon<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2022/semaphore-0.3-and-1.0/#fn1" id="fnref1">[1]</a></sup>.</p>
<p>My team has been using Semaphore in production for almost a year now, and it's been working great <em>with one annoying exception</em>. The way the rules engine is implemented means that in order to accomplish a true combination of "AND" and "OR" rules for a single feature, we ended up hoisting the OR implementation out a layer into what could be considered "userland" code. There's currently no way to do it inside of semaphore.</p>
<p>Long story short, we're tired of not having a good way to mix AND and OR support together, so I'm fixing that flaw; and it involves some breaking changes.</p>
<p>Unless you're using Semaphore and you need to plan for the upcoming data structure changes, you can probably stop reading here. The rest is very technical and will be meaningless unless you care about implementing this specific project, or I guess if you have an interest in rules engines.</p>
<p>Either way, you've been warned. :)</p>
<hr />
<h3 id="the-ultra-short-version" tabindex="-1">The ultra-short version</h3>
<p>Specifying rules in v0.2:</p>
<pre class="language-js"><code class="language-js"><span class="token punctuation">{</span><br /> <span class="token literal-property property">matchRules</span><span class="token operator">:</span> <span class="token string">'ALL'</span><span class="token punctuation">,</span> <span class="token comment">//ALL = AND, ANY = OR</span><br /> <span class="token literal-property property">rules</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token constant">RULE</span><span class="token punctuation">]</span><br /><span class="token punctuation">}</span></code></pre>
<p>Specifying rules in v1.0+:</p>
<pre class="language-js"><code class="language-js"><span class="token punctuation">{</span><br /> <span class="token literal-property property">rules</span><span class="token operator">:</span> <span class="token punctuation">[</span><br /> <span class="token punctuation">[</span><span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token comment">/* and */</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token comment">/* and */</span> <span class="token constant">RULE</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token comment">/* or */</span><br /> <span class="token punctuation">[</span><span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token comment">/* and */</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token comment">/* and */</span> <span class="token constant">RULE</span><span class="token punctuation">]</span><br /> <span class="token punctuation">]</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<hr />
<h3 id="the-full-explanation" tabindex="-1">The full explanation</h3>
<p>Consider these rules: Feature X should be enabled if:</p>
<ol>
<li>env == <code>development</code>; OR</li>
<li>(env == <code>QA</code> AND customer == <code>Disney</code>); OR</li>
<li>(env == <code>production</code> AND customer in <code>Disney, Pop Tarts</code>)</li>
</ol>
<p>Semaphore v0.2 allows you to create a flat list of rules, and a single combination expression (AND vs. OR). Because of this, you can't have different subsets of rules for different environments, for example. You can have a flag that will be on when <code>env=development</code>; and you can have a flag that will be on when <code>env=production AND customer in list "Disney, Pop Tarts"</code>... But you can't use both AND and OR in a single flag.</p>
<p>So, what we did on our team was wrap semaphore in a <code>featureFlagService</code> which allowed us to check multiple flags at once, and return true if any of those flags evaluated to true. This allowed us to create separate flags, i.e. <code>some-feature-dev</code>, <code>some-feature-qa</code>, and <code>some-feature-prod</code>, each of them implementing only 1 of the numbered sets of rules above, and then we would check all of those flags in our code:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">flagEnabled</span><span class="token punctuation">(</span><span class="token string">'some-feature-dev,some-feature-qa,some-feature-prod'</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">newImplementation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token function">oldImplementation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>This works fine, but it's annoying, because it requires us to manage 3 feature flags for every feature because we usually have 3 possible OR-cases. If you have more OR's, then you'd need even more flags!</p>
<p>Fixing this requires making a breaking change to the DSL for specifying flag rules. If you have some sort of GUI for creating and managing flags, it will need to be updated to read and output these changes to the underlying data.</p>
<p>Currently, a flag consists of:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token string-property property">'flag-id'</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">name</span><span class="token operator">:</span> string<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">description</span><span class="token operator">:</span> string<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">active</span><span class="token operator">:</span> boolean<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">baseState</span><span class="token operator">:</span> boolean<span class="token punctuation">,</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token literal-property property">matchRules</span><span class="token operator">:</span> <span class="token constant">ANY</span> <span class="token operator">|</span> <span class="token constant">ALL</span><span class="token punctuation">,</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token literal-property property">rules</span><span class="token operator">:</span> <span class="token punctuation">[</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token constant">RULE</span><span class="token punctuation">,</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token constant">RULE</span><span class="token operator">...</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">]</span></mark><br /><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>and RULEs consist of:</p>
<pre class="language-js"><code class="language-js"><span class="token punctuation">{</span><br /> <span class="token literal-property property">type</span><span class="token operator">:</span> filter <span class="token operator">|</span> <span class="token operator">%</span> <span class="token operator">|</span> nobody <span class="token operator">|</span> everybody<span class="token punctuation">,</span><br /> <span class="token literal-property property">attribute</span><span class="token operator">:</span> string<span class="token punctuation">,</span><br /> <span class="token literal-property property">operator</span><span class="token operator">:</span> <span class="token operator">==</span> <span class="token operator">|</span> <span class="token operator">!=</span> <span class="token operator">|</span> <span class="token operator"><</span> <span class="token operator">|</span> <span class="token operator"><=</span> <span class="token operator">|</span> <span class="token operator">></span> <span class="token operator">|</span> <span class="token operator">>=</span> <span class="token operator">|</span> <span class="token keyword">in</span> <span class="token operator">|</span> has<span class="token punctuation">,</span><br /> <span class="token literal-property property">comparator</span><span class="token operator">:</span> boolean <span class="token operator">|</span> numeric <span class="token operator">|</span> string <span class="token operator">|</span> string<span class="token punctuation">[</span><span class="token punctuation">]</span><br /><span class="token punctuation">}</span></code></pre>
<p>In order to support combinations of AND/OR, the flag structure is changing to the following:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token string-property property">'flag-id'</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">name</span><span class="token operator">:</span> string<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">description</span><span class="token operator">:</span> string<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">active</span><span class="token operator">:</span> boolean<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">baseState</span><span class="token operator">:</span> boolean<span class="token punctuation">,</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token literal-property property">rules</span><span class="token operator">:</span> <span class="token punctuation">[</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">[</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token operator">...</span> <span class="token punctuation">]</span><span class="token punctuation">,</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">[</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token operator">...</span> <span class="token punctuation">]</span><span class="token punctuation">,</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">]</span></mark><br /><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Note that the <code>matchRules</code> property is gone. It will now always behave as an OR against the top-level arrays, and an AND for the rules inside each inner array.</p>
<p>Hopefully that makes sense. Basically:</p>
<pre class="language-js"><code class="language-js"><span class="token literal-property property">rules</span><span class="token operator">:</span> <span class="token punctuation">[</span><br /> <span class="token punctuation">[</span><span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token comment">/* and */</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token comment">/* and */</span> <span class="token constant">RULE</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token comment">/* or */</span><br /> <span class="token punctuation">[</span><span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token comment">/* and */</span> <span class="token constant">RULE</span><span class="token punctuation">,</span> <span class="token comment">/* and */</span> <span class="token constant">RULE</span><span class="token punctuation">]</span><br /> <span class="token comment">/* or... */</span><br /><span class="token punctuation">]</span><span class="token punctuation">;</span></code></pre>
<p>This exact same structure will be required in ALL cases. That includes scenarios where you have just two rules and you want them to be evaluated using an OR:</p>
<pre class="language-js"><code class="language-js"><span class="token literal-property property">rules</span><span class="token operator">:</span> <span class="token punctuation">[</span><br /> <span class="token punctuation">[</span><span class="token keyword">this</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token comment">/* or */</span><br /> <span class="token punctuation">[</span>that<span class="token punctuation">]</span><br /><span class="token punctuation">]</span><span class="token punctuation">;</span></code></pre>
<p>... Or even just a single rule:</p>
<pre class="language-js"><code class="language-js"><span class="token literal-property property">rules</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">[</span>other<span class="token punctuation">]</span><span class="token punctuation">]</span><span class="token punctuation">;</span></code></pre>
<p>This approach is still a little bit naïve, but what it lacks in cleverness it repays in simplicity. So yes, it's true that you won't be able to have rules with deeply nested AND/OR expressions, but if you're willing to suffer some (probably minor) duplication, it should solve for every possible need.</p>
<p>I expect that this change should greatly improve my teams happiness using Semaphore<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2022/semaphore-0.3-and-1.0/#fn2" id="fnref2">[2]</a></sup>, and hopefully yours too.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>Tentatively targeting May 19th; exactly 1 year after Semaphore's <a href="https://adamtuttle.codes/blog/2021/introducing-semaphore/">introduction</a>. <a href="https://adamtuttle.codes/blog/2022/semaphore-0.3-and-1.0/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>To be fair, we're already quite happy with it! <a href="https://adamtuttle.codes/blog/2022/semaphore-0.3-and-1.0/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Thoughts on Working at the Same Company for 10+ Years2022-03-02T09:00:00Zhttps://adamtuttle.codes/blog/2022/thoughts-on-ten-years-at-the-same-company/<p><img src="https://adamtuttle.codes/img/2022/henley-design-studio-XbZgARqXROc-unsplash.jpg" alt="Baby chowing down on birthday cake" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@henleydesign?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Henley Design Studio</a> on <a href="https://unsplash.com/?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>I've been working for <a href="https://www.alumniq.com/">AlumnIQ</a> since March 2nd 2012. Today is my 10 year work anniversary.</p>
<p>In 2020 on the occasion of my work anniversary – just a few days before the pandemic got real here in the U.S.! 😬 – I wrote about how the most common advice in tech is to quit your job every couple of years to get a significant salary raise, but that somehow I had managed to <a href="https://adamtuttle.codes/blog/2020/challenge-breeds-stability/">stay in the same place for 8 years</a>. I attributed that to finding success in products, and the new and interesting challenges that brings with it.</p>
<p>Two years on and not much has changed, perspective-wise. I still love what I get to do every day because my job isn't, "Design and implement another CRUD form for another database table;" it's, "This feature has been so popular that the naive initial design is becoming a problem. Make it more resilient."</p>
<p>These types of challenges are great opportunities to think creatively, and they're hard problems to solve.</p>
<p>We continue to have all of the best problems you could hope for. Our products and services are selling well, our customers are the right mix of happy and demanding of new features, and we all enjoy working together. Occasionally users push the limits of what the software is capable of, and this is an opportunity to design and build something even more capable, or to help them fall into the pit of success.</p>
<p>This is exactly the kind of work I most enjoy: Avoid premature optimization (see also: <a href="https://adamtuttle.codes/blog/2022/design-naively/">design naïvely</a>), and then when the seams start to buckle, <em>make that part of the system more complex to support its increased demand.</em></p>
<p>The team is <strong>really</strong> small, and even that has its benefits. I've worked at organizations that had 10 people just on my team. AlumnIQ is 5 full time employees plus a few seasonal helpers for on-site operations when we do in-person events.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2022/thoughts-on-ten-years-at-the-same-company/#fn1" id="fnref1">[1]</a></sup></p>
<p>Just 3 of those 5 people are on what I would consider "my team" and 1 of those 3 is me. It means we're all on-call every 3 weeks, which is a little bit of a bummer to be honest, but it's also in large part responsible for the fact that most of the time we go from a need, to coded, to reviewed, to shipped in hours or days, rather than weeks or god forbid, months. Meetings are kept to an absolute minimum because you're not just wasting 2-3 people's time, <strong>every added person attending that meeting is burning an additional 20% of our ability to be productive in that moment.</strong></p>
<p>It's kind of a point of pride that we make a big impact with a small team.</p>
<p>I had to schedule a meeting with a couple of customers to discuss a change we'd like to make, and even just the knowledge that I'm going to have to wait a business-day or two (weekends, ugh!) to get some face time and my questions answered is more friction than I'm used to, and it was startling how odd it felt.</p>
<p>The idea of having more people to share the workload (<em>and the on-call schedule!</em>) with is appealing, but the idea of the bureaucracy and politics and red tape and ... that comes with working in a big company again makes my stomach turn.</p>
<p>I've been working from home full time for 10 years. It took some getting used to (as you all now probably know), but it's become so normal that I can't imagine going back to an office full time. If I did, I might need to bring my dog, cats, and my fireplace with me. I don't think that work-from-home really works without actual <em>trust</em> from the top of the company, so that's been another fringe benefit, too.</p>
<p>Could I have doubled my salary by now by taking jobs I'd be less interested in and moved my family around the country a few times? Probably.</p>
<p>Would I be as happy as I am with my house, my neighborhood, the school my kids attend, my hobbies, my access to entertainment, and so on? There's no way to know for sure, but I can say that I'm quite happy with what I've got and I wouldn't gamble it.</p>
<p>I like to think of it as investing in whole-family happiness. We're close to family and friends here. My kids have a good chance at having a few friends that they know for effectively their entire lives. That's not an opportunity I had, but it's one I'm glad I can give to my children.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2022/thoughts-on-ten-years-at-the-same-company/#fn2" id="fnref2">[2]</a></sup></p>
<p>Not only that, but I now have 10 years experience building a single product from scratch and all of the really good, and really bad decisions made along the way. That's a pretty unique and really valuable experience.</p>
<p>If this article has a take-away, I guess this is it: <strong>It's ok to settle down.</strong></p>
<p>Change jobs early in your career when you feel like you've leveled up and your salary doesn't reflect that. If you like where you work at that time, ask for a promotion first. It'll be easier to do this when you're umarried or early on in your family, than after you have school-aged children and established support networks.</p>
<p>But if and when you land a job that challenges you the right amount, where you like the people, and you like the work, and there are lots of little benefits that you couldn't or wouldn't get elsewhere, it's ok to plant roots.</p>
<p>Cheers to ten more years. 🍻</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p><em>Remember in-person events? Those were the days!</em> <a href="https://adamtuttle.codes/blog/2022/thoughts-on-ten-years-at-the-same-company/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>This is not said to disparage my parents. <a href="https://twitter.com/AdamMGrant/status/1494708914922954759">They were pretty good ancestors</a> – they traded one thing for another. Giving up geographical stability to gain financial stability is an easy decision to make, and I benefitted greatly from their sacrifices. <a href="https://adamtuttle.codes/blog/2022/thoughts-on-ten-years-at-the-same-company/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Design Naïvely2022-02-10T00:00:00Zhttps://adamtuttle.codes/blog/2022/design-naively/<p><img src="https://adamtuttle.codes/img/2022/timo-volz-9Psb5Q1TLD4-unsplash.jpg" alt="A very complex intersection seen from a high vantage point" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@magict1911?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Timo Volz</a> on <a href="https://unsplash.com/s/photos/complex?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>What exactly does it mean to avoid premature optimization when writing software?</p>
<p>When building a new town from scratch, don't plan for hi-rise buildings, complex freeway interchanges and underground mass transit lines. Start with the buildings and roads and parking lots that will attract people to your new town.</p>
<p>Which is worse?</p>
<ul>
<li>Spending 2x-3x the initial investment of time and money to demolish and rebuild infrastructure to support the town as it grows</li>
<li>Spending 20x the minimal investment of time and money to build a city that could support a thriving community of thousands, but nobody shows up</li>
</ul>
<p>Build just enough of your application to say you have the feature.</p>
<p>Do it the easy way.</p>
<p>Don't care that it won't work if you achieve Facebook-activity-level scale. That level of activity almost never shows up overnight.</p>
<p>Odds are good that you'll never need to.</p>
<p>And that's a good thing.</p>
<p>It freed you up to spend more time building more new ideas (naïvely). The more you build, the more likely one of those ideas is to find success.</p>
<p>So put it in the monolith. Use algorithms that are easy to implement. Avoid complex workflows like message queues and service busses.</p>
<p>Watch activity logs. Measure performance. When, <em><strong>and only when</strong></em> it's being stressed near to its breaking point so you have to closely monitor the feature during heavy usage periods to make sure it doesn't explode and ruin your day, and/or when those processes become critical-path to the success of your business, should you invest time and money to add complexity and support increased demand.</p>
Keeping the Game Alive After Google Takes Their Ball and Goes Home2022-01-28T00:00:00Zhttps://adamtuttle.codes/blog/2022/keeping-the-game-alive-google-takes-their-ball-and-goes-home/<p class="photo-byline"><img src="https://adamtuttle.codes/img/2022/christopher-campbell-syyBwqVX0Hc-unsplash.jpg" alt="" />
Photo by <a href="https://unsplash.com/@chrisjoelcampbell?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Christopher Campbell</a> on <a href="https://unsplash.com/?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>Long ago, Google allowed anyone with the know-how to create a google account using a custom domain name. My website was previously <code>fusiongrokker.com</code> and the <code>adam@</code> email address was my google account. <a href="https://support.google.com/a/answer/2855120?hl=en">As of December 6th 2012</a>, they stopped allowing new signups for this free service, but existing accounts were grandfathered and remained free.</p>
<p>Earlier this year, Google announced that they're no longer going to be allowing the freeloaders to stick around for free. On July 1st free accounts for GSuite, including Google Docs, GMail, Google Drive, and more <a href="https://www.theverge.com/2022/1/19/22891509/g-suite-legacy-free-google-apps-workspace-upgrade">will become past-tense</a>. If you like, you can choose to stick around for $6 per user per month. In fact, you have to pick a plan by May 1st. The clock is ticking. GMail, Drive, Docs, etc remain free as long as you don't mind using an <code>@gmail.com</code> email address.</p>
<h2 id="not-sure-if-you're-affected%3F" tabindex="-1">Not sure if you're affected?</h2>
<p><img src="https://adamtuttle.codes/img/2022/google_account.png" alt="Google account selector, showing that Adam Tuttle is logged in using an @gmail.com email address" style="float: right;margin-left:15px;margin-bottom:15px;max-width:300px;" /> I had a friend who wasn't sure if they were affected because they forward their email address at their custom domain to their gmail. If that's you, you should be fine.</p>
<p>You can confirm a couple of ways. Think about whatever service you're worried about losing your data in. Let's say Google Drive. Log into Drive and then click your profile picture in the upper-right. If it's an <code>@gmail.com</code> address, you're fine. If it's your custom domain, you're affected. Keep reading.</p>
<p>The other thing that you can do to confirm is to check the DNS settings for your domain. Here are mine:</p>
<p><img src="https://adamtuttle.codes/img/2022/fusiongrokker_dns.png" alt="DNS configuration for fusiongrokker.com, showing MX records pointing to various googlemail.com domains" /></p>
<p>Note that I have MX records (for email) and they are all pointing to various subdomains of <code>google.com</code> and <code>googlemail.com</code>. This is a necessary step for setting up your domain to use GMail. If you don't have something similar, it's <em>almost</em> guaranteed that you're not affected.</p>
<p>I'm pretty sure that using GMail for your domain isn't a requirement to use that email address as a google account, but chances are really good that if you wanted this account it was for the amazing GMail interface compared to the junk that was available as alternatives at the time.</p>
<h2 id="it's-for-the-better" tabindex="-1">It's for the better</h2>
<p>The fact of the matter is that shortly after they discontinued signups for new free domains accounts, those of us that stuck around started to feel unwanted anyway. Many services and features didn't work well, if at all, with custom domains. A few years back I got fed up with this and created a new <code>@gmail.com</code> to become my new primary account. But I had been using my <code>@fusiongrokker.com</code> email address as my primary since 2008, probably something like 10 years. It's not so easy to update every person, and worse, every online account that uses one email address to use the new one. And I'm lazy.</p>
<p>From my point of view, we (freeloaders) have no right to complain. They've provided some pretty great services for completely free (minus all the data they scraped out of our accounts) for a long damn time. I'm not surprised it's ending. I'm a little surprised and put-off by the short timeline.</p>
<p>Anyway, I'm a fairly heavy google-accounts user, so I'm going to have to figure a few things out and I figured it might be helpful to document the processes I'm using to keep my digital life moving forward. Here are the services that I use on the regular, and what I'm doing to deal with the loss of my free account:</p>
<h3 id="gmail" tabindex="-1">GMail</h3>
<p>Fortunately enough, I was already on the path to migrating to an <code>@gmail.com</code> email address, thanks to needing it for things like YouTube Premium and Family Link (for controlling my kids' accounts), neither of which support custom domains. My blast radius for email is limited to:</p>
<ul>
<li>Setting up an autoresponder to notify anyone that emails me about my new email address</li>
<li>Updating all three billion or so accounts I have online to use the new email address. Thankfully most of them will be easy to locate thanks to my long-time usage (and advocation) of 1Password. Still going to be tedious, but at least possible!</li>
<li>There are definitely some accounts out there that use a username instead of email address, and thus don't pop out in my searches for the old email address in 1Password. For that reason, I intend to continue paying for the domain for at least another year, and <a href="https://aws.amazon.com/blogs/messaging-and-targeting/forward-incoming-email-to-an-external-destination/">using Amazon SES to forward the emails</a> it receives to my new address.</li>
</ul>
<h3 id="google-calendar" tabindex="-1">Google Calendar</h3>
<p>This was originally the one causing me the most stress. I have several google calendars, some personal and private and some that I share with family. As it turns out it's quite simple to export and import your calendar data. You can export all calendars in bulk, or one calendar at a time, and importing them is as easy as going into <strong>Settings > Import & export</strong>. When importing you need to have an existing calendar to copy the events into. Just to be sure I would be happy with it, I started by importing into a new temp calendar I created for the purposes of testing. After looking at the result to confirm I was happy with it, I trashed that calendar and repeated the process with the real calendars. It took seconds of work (couched in minutes of anxiety).</p>
<h3 id="google-voice" tabindex="-1">Google Voice</h3>
<p>My primary phone number (as far as everyone that has my phone number is concerned) is a number provided through Google Voice, attached to my <code>@fusiongrokker.com</code> account. So not only do I need to vacate that email address, I also need to update my phone number everyhwere, too. There is an option to <a href="https://support.google.com/voice/answer/1065667#googlexfer">transfer your number to a different google account</a>, if you're happy with Google Voice, but I'm not. If I'm going to go through some trouble here I might as well put in a little extra effort to end up with the better experience.</p>
<p>Google Voice is <em>fine</em>, but the native Messages app on Android is <em>better</em> and at least for the moment seems to be where Google is most likely to continue to invest in development. Voice seems to have fallen into the same void that GTalk, GChat, Google Wave, and other Google messaging apps disappeared into. I will be sad to lose the ability to make and receive voice calls through my web browser on my laptop, but it'll be a small price to pay for a UI that doesn't freeze up because you left the tab open for 24 hours. Not to mention the Google Assistant (Android phones, Android Auto, Google Home devices, etc) has no idea what Google Voice is. So hopefully I'll gain the ability to send and receive text messages through that, too.</p>
<h3 id="google-photos" tabindex="-1">Google Photos</h3>
<p>I really like Google Photos. The "AI" built into searching is really nice. I'm not sure yet if I want to give that up.</p>
<p>On the other hand, we recently added a NAS to our household network (<a href="https://amzn.to/33TFCf9">a Synology DS420+</a>) and it has a similar feature to automatically backup photos and videos from your phone. Adding my wife and kids' photos and videos to the collection automatically sounds pretty great. I'm sure it won't be as nice or as handy as Google Photos. I also occasionally share things with friends and family and that would make it a little harder.</p>
<p>Either way, getting your photos is as easy as going to <a href="https://takeout.google.com/">Google Takeout</a>, make sure you're signed into the correct google account (check in the upper-right corner), select Photos and a few export options (notify me by email, split zips at 2gb, etc), and wait for them to email you download links.</p>
<p>I don't think Photos has any sort of bulk-import that preserves the original file date. If you tried to re-upload the files you get back from Takeout, I expect they'll be marked as created on their upload date.</p>
<h3 id="gsuite-(docs%2C-drive%2C-sheets%2C-etc)" tabindex="-1">GSuite (Docs, Drive, Sheets, etc)</h3>
<p>Just like Photos, you can export your entire Google Drive contents from <a href="https://takeout.google.com/">Google Takeout</a>. They give you options on how you want each file type to be converted, for example your Google Docs documents could be converted to PDF or DOCX. After selecting Drive from the list of service data you want to export, click on the "Multiple Formats" button to see the options of how you want your files converted.</p>
<h3 id="takeout-for-the-rest" tabindex="-1">Takeout For the Rest</h3>
<p>There's a really good chance that <a href="https://takeout.google.com/">Google Takeout</a> covers everything else.</p>
How I Reduce Duplication in Tests2022-01-25T00:00:00Zhttps://adamtuttle.codes/blog/2022/how-i-reduce-duplication-in-tests/<p><img src="https://adamtuttle.codes/img/2022/phil-shaw-zAZYuch7deE-unsplash.jpg" alt="Storm Troopers from Star Wars" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@phillshaw?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Phil Shaw</a> on <a href="https://unsplash.com/s/photos/clone?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>When writing tests for a complex bit of logic that has many test cases, I often find myself passing slight variations on input data to a method.</p>
<p>Here's the type of thing I am using it for, cleaned up for sharing in public. I have a function that makes a database query, which behaves in a lot of different possible ways depending on what data is returned. Since we're testing communication across an I/O boundary it makes sense to mock that so that our unit/integration tests can run quickly. So here's one sample record, as mocked for returning for the first query to be executed. This is using <a href="https://jestjs.io/">jest</a> syntax for mocking, if you're not familiar with it.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> mockResponse <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">personId</span><span class="token operator">:</span> <span class="token number">123</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">dateTimeCreated</span><span class="token operator">:</span> <span class="token string">'2021-01-11 06:06'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">email</span><span class="token operator">:</span> <span class="token string">'foo@bar.com'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">firstName</span><span class="token operator">:</span> <span class="token string">'John'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">lastName</span><span class="token operator">:</span> <span class="token string">'Smith'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">hasPlumbus</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">isRegistered</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">hasDonated</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">lifetimeDonationSumCents</span><span class="token operator">:</span> <span class="token number">10000</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br />db<span class="token punctuation">.</span>query<span class="token punctuation">.</span><span class="token function">mockImplementationOnce</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">[</span>mockResponse<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>In reality, the objects I'm using this technique on are easily twice as long as that example.</p>
<p>Now, for the purposes of this discussion, let's assume that you need to test 100 different possible variations of this data handler. The naive approach is to copy and paste the mock data as-is for all 100 tests. And there's nothing inherently wrong with that approach, except that it makes your test files that much larger.</p>
<p>Instead, I like to use the JavaScript <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax">spread operator</a> to limit what needs to be included in each test to only the parts that affect that test.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> mockRow <span class="token operator">=</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">personId</span><span class="token operator">:</span> <span class="token number">123</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">dateTimeCreated</span><span class="token operator">:</span> <span class="token string">'2021-01-11 06:06'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">email</span><span class="token operator">:</span> <span class="token string">'foo@bar.com'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">firstName</span><span class="token operator">:</span> <span class="token string">'John'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">lastName</span><span class="token operator">:</span> <span class="token string">'Smith'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">hasPlumbus</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">isRegistered</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">hasDonated</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">lifetimeDonationSumCents</span><span class="token operator">:</span> <span class="token number">10000</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'throws for a missing email address'</span><span class="token punctuation">,</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">const</span> testData <span class="token operator">=</span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token operator">...</span>mockRow<span class="token punctuation">,</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token literal-property property">email</span><span class="token operator">:</span> <span class="token keyword">null</span></mark><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> db<span class="token punctuation">.</span>query<span class="token punctuation">.</span><span class="token function">mockImplementationOnce</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">[</span>testData<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> <span class="token keyword">await</span> <span class="token function">expect</span><span class="token punctuation">(</span><span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> app<span class="token punctuation">.</span><span class="token function">workOnPerson</span><span class="token punctuation">(</span><span class="token number">123</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">.</span>rejects<span class="token punctuation">.</span><span class="token function">toThrow</span><span class="token punctuation">(</span><span class="token string">'Email is null'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>In the highlighted lines you can see that I'm creating a new <code>testData</code> object, which inherits all of the properties of the <code>mockRow</code> object, but then overwrites the <code>email</code> property to null, because that's the only one that matters for the purpose of this test. This is extremely useful when you need the entire record to contain something realistic so that you're isolating the one thing you want to test, without duplicating that realistic data between every test.</p>
<p>Why is this important? Well, like I said, without it your test code could potentially grow at an alarming rate. For my current project, even while making heavy use of this technique, I found myself with more than 9 lines of test code (including mock data and mock modules) for every line of application code. Don't take that to mean testing is bad or wasteful, but testing well can be a lot of work and require a lot of setup data.</p>
TIL: TypeScript Index Signatures Are Super Useful!2022-01-14T00:00:00Zhttps://adamtuttle.codes/blog/2022/til-typescript-object-indexing/<p><img src="https://adamtuttle.codes/img/2022/markus-spiske-vPGJ2fMsWkQ-unsplash.jpg" alt="A ripe banana on a yellow background" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@markusspiske?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Markus Spiske</a> on <a href="https://unsplash.com/s/photos/definition?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<blockquote>
<p><em><strong>Heads up:</strong> I'm new to TypeScript. This is my first entry on the subject and it's entirely possible I'm giving bad advice here. I'm sharing as I learn, and I haven't (yet?) taken the time to read through the <a href="https://www.typescriptlang.org/docs/handbook/intro.html">TypeScript handbook</a>. Please feel free to help me learn more, and definitely don't take what I say here as any sort of best practice; even if I do get lucky enough to stumble into the truth.</em></p>
</blockquote>
<p>I ran into a problem trying to type some data that represented a collection of other Types that I had massaged into a different shape in order to be useful for my application. It took some googling and reading, but eventually I found the <a href="https://www.typescriptlang.org/docs/handbook/2/objects.html#index-signatures">index signatures</a> documentation, which solved my problem.</p>
<p>Let's assume you have an array of Widget objects:</p>
<pre class="language-ts"><code class="language-ts"><span class="token keyword">interface</span> <span class="token class-name">Widget</span> <span class="token punctuation">{</span><br /> category<span class="token operator">:</span> <span class="token string">'thingamajigs'</span> <span class="token operator">|</span> <span class="token string">'whosawhatsis'</span><span class="token punctuation">;</span><br /> squanches<span class="token operator">:</span> <span class="token builtin">boolean</span><span class="token punctuation">;</span><br /> universeId<span class="token operator">:</span> <span class="token builtin">string</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>And let's further assume that you need to break that array of <code>Widget</code>s up into a set of named arrays, one per category, and the category name is the key of the object. After doing so, it would look like this:</p>
<pre class="language-js"><code class="language-js"><span class="token punctuation">{</span><br /> <span class="token string-property property">'thingamajigs'</span><span class="token operator">:</span> <span class="token punctuation">[</span><br /> <span class="token comment">//Wiget</span><br /> <span class="token comment">//Wiget</span><br /> <span class="token comment">//Wiget</span><br /> <span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token string-property property">'whosawhatsis'</span><span class="token operator">:</span> <span class="token punctuation">[</span><br /> <span class="token comment">//Wiget</span><br /> <span class="token comment">//Wiget</span><br /> <span class="token punctuation">]</span><br /><span class="token punctuation">}</span></code></pre>
<p>The function that creates this data structure is pretty simple<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2022/til-typescript-object-indexing/#fn1" id="fnref1">[1]</a></sup>:</p>
<pre class="language-ts"><code class="language-ts"><span class="token keyword">function</span> <span class="token function">categorizeWidgets</span><span class="token punctuation">(</span>widgets<span class="token operator">:</span> Widget<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">:</span> <span class="token builtin">any</span> <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> groups<span class="token operator">:</span> <span class="token builtin">any</span> <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span><br /> widgets<span class="token punctuation">.</span><span class="token function">forEach</span><span class="token punctuation">(</span><span class="token punctuation">(</span>w<span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> cat <span class="token operator">=</span> w<span class="token punctuation">.</span>category<span class="token punctuation">;</span><br /> groups<span class="token punctuation">[</span>cat<span class="token punctuation">]</span> <span class="token operator">??=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /> groups<span class="token punctuation">[</span>cat<span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span>w<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> groups<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>I've used <code>any</code> here as a placeholder for this imaginary anonymous type that we need to create that defines what we're trying to produce. So my question was, "How do I create a type that indicates that it's an object with arbitrary strings as keys, and arrays of <code>Widget</code>s as values?"</p>
<p>If we take off those <code>any</code> types, we get this error from TypeScript:</p>
<blockquote>
<p>Element implicitly has an 'any' type because expression of type 'string' can't be used to index type '{}'.</p>
</blockquote>
<p>We can use the <code>any</code> type and call it a day (hey, the error went away!), but that has deleterious effects downstream in the app, since TypeScript won't really know what to expect, and so it's not really offering us much (any? (heh)) benefit when working with this data after it's been massaged. So it would be nice if we knew how to properly type this.</p>
<p>We already have the <code>Widget</code> type, so you might think that you should create a <code>CategorizedWidgetCollection</code> type —and maybe you should, depending on how often you'll use it— but regardless, how do you specify the type of this object?</p>
<p>I started with the basic <code>{}</code> (or <code>Object</code>), but of course TypeScript wasn't too happy with me. The same error comes back.</p>
<p>So here's the answer, given a name only to help it syntax highlight and make sense:</p>
<pre class="language-ts"><code class="language-ts"><span class="token keyword">interface</span> <span class="token class-name">CategorizedWidgetCollection</span> <span class="token punctuation">{</span><br /> <span class="token punctuation">[</span>index<span class="token operator">:</span> <span class="token builtin">string</span><span class="token punctuation">]</span><span class="token operator">:</span> Widget<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>And here's how you might apply it in an anonymous-type fashion:</p>
<pre class="language-ts"><code class="language-ts"><span class="token keyword">function</span> <span class="token function">categorizeWidgets</span><span class="token punctuation">(</span>widgets<span class="token operator">:</span> Widget<span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token punctuation">[</span>index<span class="token operator">:</span> <span class="token builtin">string</span><span class="token punctuation">]</span><span class="token operator">:</span> Widget<span class="token punctuation">[</span><span class="token punctuation">]</span> <span class="token punctuation">}</span> <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> groups<span class="token operator">:</span> <span class="token builtin">any</span> <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span><br /> widgets<span class="token punctuation">.</span><span class="token function">forEach</span><span class="token punctuation">(</span><span class="token punctuation">(</span>w<span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> cat <span class="token operator">=</span> w<span class="token punctuation">.</span>category<span class="token punctuation">;</span><br /> groups<span class="token punctuation">[</span>cat<span class="token punctuation">]</span> <span class="token operator">??=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /> groups<span class="token punctuation">[</span>cat<span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span>w<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> groups<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Note that the first line of the method still declares the <code>groups</code> object as <code>any</code> but TS doesn't complain about this because it can see from the input type, the output type, and the logic of the function that the contract is being adhered to.</p>
<p>So the trick was the use of the <code>[index: string]</code> syntax. You're telling TS that the object will be indexed by strings. If we look back at the error message one last time...</p>
<blockquote>
<p>... because expression of type 'string' can't be used to index type '{}'.</p>
</blockquote>
<p>Hopefully, for myself and for you alike, seeing this error message in the future will jog our memories about this <code>[index:]</code> syntax.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>If you're not familiar with the syntax <code>x ??= y</code>, it's shorthand for <code>if (typeof x === 'undefined' || x === null){ x = y }</code>. I only just <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Logical_nullish_assignment">learned</a> of it today. This is something I've been wanting for at least 10 years! I love how much JS is evolving these days! <a href="https://adamtuttle.codes/blog/2022/til-typescript-object-indexing/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Breaking: News Organizations Over-Hype Small Problems2022-01-10T00:00:00Zhttps://adamtuttle.codes/blog/2022/breaking-news-organizations-over-hype-small-problems/<p><img src="https://adamtuttle.codes/img/2022/ahmad-kanbar-cLXoMUtGi0k-unsplash.jpg" alt="Mole" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@ahmad_kanbar?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">ahmad kanbar</a> on <a href="https://unsplash.com/s/photos/mole?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>There has been no shortage of articles in the last few days sensationalizing the fact that there's been yet-another incident on npm where someone's brain has been taken over by parasites and they've sabotaged modules they own in order to make some sort of political statement or just to be a jerk (see also: <a href="https://github.com/left-pad/left-pad/issues/4">left-pad</a>)...</p>
<p>Look at the headline for <a href="https://www.bleepingcomputer.com/news/security/dev-corrupts-npm-libs-colors-and-faker-breaking-thousands-of-apps/">this BleepingComputer article</a>:</p>
<blockquote>
<h4 id="dev-corrupts-npm-libs-'colors'-and-'faker'-breaking-thousands-of-apps" tabindex="-1">Dev corrupts NPM libs 'colors' and 'faker' breaking thousands of apps</h4>
</blockquote>
<p>I suppose to normies who don't understand encryption from hashing from corruption, this might be a sufficient headline. <em>It's a bunch of weird characters that don't make any sense to me, so it must be corrupted!</em></p>
<p>What actually happened? The developer published new versions of two of their popular libraries, and these new versions were intentionally broken and in lieu of doing their job, they wrote the word "liberty" a bunch of times and lots of odd-looking characters to the screen; some reports indicating that this happened in an infinite loop.</p>
<p>But did the developer "corrupt" the whole module? Nope. It's just a new version.</p>
<p>Functionally this is <em>no different</em> from releasing a new version with a bug that's so bad the library is completely useless. It's not often that bugs that bad hit the wild, but it happens. And, as the same BleepingComputer article notes, in graf 40 (of 40):</p>
<blockquote>
<p>In the meantime, users of 'colors' and 'faker' NPM projects should ensure they are not using an unsafe version. Downgrading to an earlier version of colors (e.g. 1.4.0) and faker (e.g. 5.5.3) is one solution.</p>
</blockquote>
<p>Yup, just use the previous version.</p>
<h2 id="responsible-software-development" tabindex="-1">Responsible Software Development</h2>
<p>I think as an industry we need to start taking more responsibility for our work. There are tools, <em>espcially for the github+npm ecosystem</em> that make it really easy to prevent this sort of thing from happening.</p>
<ol>
<li>Libraries should version-lock their dependencies. Instead of specifying that anything listed as <a href="https://docs.npmjs.com/cli/v8/configuring-npm/package-json#dependencies">semver-compatible</a> with version 2.0.0 is ok (e.g. <code>"isarray":"^2.0.0"</code>), which seems to be the default posture, they should specify the exact version of the dependency that has been <strong>verified to be working</strong>. I'm not the first to point any of this out. <a href="https://stackoverflow.com/questions/22343224/whats-the-difference-between-tilde-and-caret-in-package-json#comment45166739_22345808">The top comment on the top answer for the top Stack Overflow question about usage of <code>~</code> vs <code>^</code> in dependency version specification</a> makes the same plea: "DO NOT BLINDLY ACCEPT DOWNSTREAM DEPENDENCIES."</li>
<li>Use a tool like <a href="https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependencies-updated-automatically/about-dependabot-version-updates">dependabot</a> to stay aware of new releases of packages that you depend on.</li>
<li>Use automated tests and continuous integration tools like <a href="https://github.com/features/actions">GitHub Actions</a> to automatically test the new dependency version for compatibility with your library.</li>
<li>If the automated tests pass, <a href="https://github.com/ahmadnassri/action-dependabot-auto-merge">auto-merge the pull request</a>.</li>
</ol>
<p>I do all of these things for this website (except #1, because it's not a library). And when those PR's are auto-merged after the tests pass, the website is automatically re-deployed. The dependency graph of this website keeps itself completely up to date in an entirely automated fashion, and if something were to make the tests fail, the pull request would sit there open as a nag for me to take a look at it.</p>
<p>All of these things I've listed above have extremely generous free tiers. In fact I think the only one that might eventually charge you is Github Actions, which gives you 3,000 minutes per month for free. If your project got 10 updates per day, running the tests would have to average > 9.6 minutes per run before you'd run out of free minutes, even in a month with 31 days. Simply put: not very likely to happen.</p>
<h2 id="conclusion" tabindex="-1">Conclusion</h2>
<p>Did some stuff break? Yup. Here on the internet we call that "a day that ends in -y".</p>
<p>Get used to it.</p>
<p>Learn from it.</p>
<p>Do better.</p>
Automating CFML Library Releases2021-12-01T00:00:00Zhttps://adamtuttle.codes/blog/2021/automating-cfml-library-releases/<p><img src="https://adamtuttle.codes/img/2021/zhang-kenny-Gx1raEg_3Zw-unsplash.jpg" alt="A man sleeps on a tree branch in a pose reminiscent of a tiger" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@kennyzhang29?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Zhang Kenny</a> on <a href="https://unsplash.com/s/photos/relaxing?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<h3 id="this-entry-was-last-updated-2022-01-15-%F0%9F%98%8E" tabindex="-1">This entry was last updated 2022-01-15 😎</h3>
<p>This message keeps coming back into my life (not that I'm complaining)...</p>
<p><a href="https://twitter.com/KentBeck/status/250733358307500032">https://twitter.com/KentBeck/status/250733358307500032</a></p>
<p>Publishing new versions of <a href="https://github.com/atuttle/taffy">Taffy</a>, my library for authoring REST API's in CFML, used to be too hard.</p>
<p>Not that it was <strong><em>hard</em></strong>, per se. But <strong><em>hard enough</em></strong> that I didn't do it as often as I would have liked.</p>
<p>I recently learned this lesson with this very website. Finally having it on a tech stack that makes it dead-simple to write and deploy new articles has meant that I'm writing much more often. (This is my 35th entry for 2021. Because of <a href="https://adamtuttle.codes/blog/2019/friction-stops-things/">the friction involved in posting</a>, I wrote 18 last year, and 13 the year before that.)</p>
<p>Compare that to <a href="https://taffy.io/">taffy.io</a> which listed version 3.1.0 (from March 2016!) as the latest version until late November of '21, whereas version 3.2.0 was released in November of 2017, and 3.3.0 was released in August of 2021. Not only that, but there were additional changes since 3.3.0 that weren't included in a new version yet, because, well... My laziness was outweighing the toil of doing a release.</p>
<h2 id="the-toil" tabindex="-1">The Toil</h2>
<p>Here's what's involved in releasing a new version of Taffy:</p>
<ul>
<li>As we merge pull requests with new features and bug fixes, we add them to the documentation, with an assumed version number (we should probably switch this to be something like <code>@next</code>, because sometimes the version number ends up changing...)</li>
<li>Eventually —as <a href="https://en.wikipedia.org/wiki/Benevolent_dictator_for_life">BDFL</a>— I decide that we've reached a good moment for a release, and do the following things, if I remember them all and/or can be bothered:
<ul>
<li>Create a new git tag with the updated version number (obeying <a href="https://semver.org/">SemVer</a>), and push it to GitHub</li>
<li>Add the (hidden) docs for the new version to the version listing so that it's no longer hidden.</li>
<li>Tweet about it on <a href="https://twitter.com/taffyio">@taffyio</a> and <a href="https://twitter.com/adamtuttle">@AdamTuttle</a>.</li>
<li>Share the news in the <code>#taffy</code> channel of the <a href="https://cfml-slack.herokuapp.com/">CFML Slack</a>.</li>
<li>Post about it on my <s>blog</s> digital garden. Not sure if I want to continue that. 🤔</li>
<li>Update the version number on the <a href="https://taffy.io/">taffy.io</a> website.</li>
</ul>
</li>
</ul>
<p>I almost forgot to include that last list item, which illustrates how annoyingly manual this whole thing is! 😂</p>
<h2 id="now-fully-automated!-%F0%9F%A4%98%F0%9F%8F%BB-%F0%9F%98%8E" tabindex="-1">Now fully automated! 🤘🏻 😎</h2>
<p>I've...</p>
<ul>
<li>
<p>added a GitHub integration to the <code>#taffy</code> Slack channel, which announces new issues, pull requests, and releases (among other things).</p>
</li>
<li>
<p>updated <a href="https://taffy.io/">taffy.io</a> to <a href="https://github.com/atuttle/Taffy/blob/5fded423c041edf4de7a3910967378c51bae809d/index.html#L74-L81">display the latest release</a> info <a href="https://github.com/atuttle/Taffy/blob/5fded423c041edf4de7a3910967378c51bae809d/index.html#L99">from the GitHub API</a> at page load. Real-time updates! Huzzah!</p>
</li>
<li>
<p>added a GitHub workflow that enforces use of the <a href="https://semver.org/">SemVer</a> labels for all Pull Requests (more on why those are required in a moment)</p>
</li>
<li>
<p>added a GitHub workflow that creates a new tag, marks it as a release, generates release notes, and copies the <code>@next.md</code> documentation file as the new version number (e.g. <code>3.4.0.md</code>) so that it's possible to browse the docs for that version no matter how far we go into the future.</p>
<ul>
<li>To keep things simple, since we already had a package.json in order to use npm-provided LessCSS tooling, it uses the <code>npm version</code> command to decide the new version number. This is the only input needed when creating a new release.</li>
<li>There's a new <code>/.github/release.yml</code> file that tells GitHub how to interpret each pull request since the last version, based on the semver labels I mentioned above. If a PR has the <code>Semver: MAJOR</code> label, it gets listed under the "Breaking Changes" heading, and so-on for <code>Semver: MINOR</code> and <code>Semver: PATCH</code>. Now release notes, which were probably the biggest contributor to my release anxiety and the primary reason for lack of releases, are 100% automated!</li>
<li>In addition, I've folded the TaffyDocs repo back into the Taffy repo, for a couple of reasons. Not only does it make things simple for this automation to copy <code>@next.md</code> to <code>{new-version}.md</code>, but it means that when contributors submit a PR, the same PR can contain the code changes and the documentation changes. *chef-kiss*</li>
</ul>
</li>
<li>
<p>And lastly, the same workflow that does all of the above, tweets when there's a new release and links to the release notes.</p>
</li>
</ul>
<p>With all of that done, the hardest part of doing a release now is keeping track of what's been merged so that I know whether I should bump the major/minor/patch version number. I've got a system for doing this manually, and the fewer updates that I include per version the easier this will be to keep track of, so I'm hopeful that releases will become more frequent and smaller, now.</p>
<p><a href="https://github.com/atuttle/Taffy/blob/main/.github/workflows/release.yml">You can see the current state of my release automation here</a>.</p>
Thoughts on "The Phoenix Project"2021-11-28T00:00:00Zhttps://adamtuttle.codes/blog/2021/thoughts-on-the-phoenix-project/<p><a href="https://amzn.to/3p1l4Yi"><img src="https://adamtuttle.codes/img/2021/phoenix-project-cover.jpg" alt="Cover art for book: The Phoenix Project" /></a></p>
<p>At the recommendation of one of my <a href="https://workingcode.dev/">podcast</a> co-hosts, <a href="https://www.bennadel.com/">Ben Nadel</a>, I read <strong><a href="https://amzn.to/3p1l4Yi">The Phoenix Project</a></strong> last week; and I thought I would share both my review of the book, and some notes that I took to refer back to later.</p>
<h2 id="my-review" tabindex="-1">My Review</h2>
<p>If it wasn't for the —frankly, <span class="bleep">fuck</span>ing offensive— callously negative attitude toward "developers" in the first few chapters of the book<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2021/thoughts-on-the-phoenix-project/#fn1" id="fnref1">[1]</a></sup>, I might have even given it 5 stars in <a href="https://www.goodreads.com/review/show/4345054208">my review</a>. Instead, I gave it 4 stars.</p>
<p>My approach to reading books is often feast or famine. If I'm really into it then I can't put it down and I'll finish it in just a couple of days. If I'm forcing myself to read it for some reason even though I'm not truly enjoying it then I might only pick it up for an hour or two per week. TPP falls into the former category. I actually stayed up past midnight (I usually go to bed around 9:30) to finish the last few chapters. So while the book had some issues, I have to say that I greatly enjoyed it.</p>
<p>And there is no doubt in my mind that I enjoyed it more than I would have enjoyed some drab "Why and How To Agile" book. Using a fictional novel to illustrate the concepts is kind of brilliant, if a touch contrived.</p>
<p>I don't think the average non-IT person would really care for it. They would probably come away wondering what makes the idea of <strong>Kanban</strong> so exciting. But for those of us working at companies that sometimes can't seem to keep from aiming a gun squarely at its own foot, it also offers some catharsis.</p>
<p>That said, <em>there be dragons here</em>. Remember that spending time setting up tools and processes that are supposed to help you <em>do more and better work</em> is a useless distraction if keeps you from <em>doing the work</em>. Put another way: it's more fun to setup processes for doing work than it is to do work. There's a trap there, so don't fall into it.</p>
<p>All in all, I'm really glad I read it and I will probably read it again. I'd like to add more detail to the notes I was able to cobble together the day after finishing the book. This was yet another example of a book that I wished I would have known how much I'd like it in advance so that I could be prepared to take notes as it happens. Alas, reading again will have to do.</p>
<p>There's also <a href="https://amzn.to/3xmuktO">a sequel</a>, which I've started reading but I'm not very far into yet. So far it seems like an Ender/Bean situation: Mostly the same timeline but told through someone else's eyes. (A developer!<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2021/thoughts-on-the-phoenix-project/#fn2" id="fnref2">[2]</a></sup>)</p>
<h2 id="my-notes" tabindex="-1">My Notes</h2>
<p>As I said previously, I intend to refer back to these notes in the future when and as they might be helpful in my career. To separate them from my review above, and also because I needed an excuse to play with Notion, I have also made <a href="https://atcodes.notion.site/The-Phoenix-Project-25659ef25d124005a4dd6d2f1d5ca292">this public notion page containing only my notes</a>.</p>
<h3 id="the-4-types-of-work" tabindex="-1">The 4 Types of Work</h3>
<ol>
<li>Business Projects</li>
<li>Internal Projects</li>
<li>Changes (bug fixes, improvements generated by the first two types)</li>
<li>Unplanned work — <strong>the most dangerous</strong></li>
</ol>
<h3 id="wip-(%22whip%22)-~-work-in-progress" tabindex="-1">WIP ("whip") ~ Work In Progress</h3>
<p>What drags everything down is clutter of unfinished projects. Keeping too many balls in the air results in more time spent keeping them all from hitting the ground than working on them.</p>
<blockquote>
<p>Formula for Wait time = % busy / % idle</p>
<ul>
<li>50% busy / 50% idle = 1 unit of wait time</li>
<li>90% busy / 10% idle = 9 units of wait time</li>
<li>99% busy / 1% idle = 99 units of wait time</li>
</ul>
</blockquote>
<h3 id="kanban-boards" tabindex="-1">Kanban Boards</h3>
<p>Visualize everything on everyone's plate. We too easily get sucked into juggling invisible tasks which causes slippage and demoralizes. Making all tasks visible creates the opportunity to prioritize and defer.</p>
<h3 id="10-deployments-per-day" tabindex="-1">10 Deployments per Day</h3>
<p>Less about actually completing enough work to <em>need</em> 10 deployments/day and more about having the agility to be able to respond that rapidly when the need arises. If you can deploy 10x daily then your deploys are automated, fast, and reliable.</p>
<h3 id="the-theory-of-constraints" tabindex="-1">The Theory of Constraints</h3>
<p>Identify bottlenecks and work to prevent a backup of WIP behind them, by:</p>
<ol>
<li>systematizing the bottleneck so more people can share the load</li>
<li>automating what can be automated</li>
<li>using proper work queues instead of treating whoever yells the loudest as a VIP</li>
</ol>
<h3 id="the-3-ways" tabindex="-1">The 3 Ways</h3>
<p>Maximize value delivery by optimizing workflows. "Value" is anything a customer finds useful.</p>
<ol>
<li><strong><em>The First Way:</em></strong> Continuously find and implement ways to improve delivery. (<a href="https://en.wikipedia.org/wiki/Kaizen">Kaizen</a>)</li>
<li><strong><em>The Second Way:</em></strong> Shorten feedback cycles to get advance warnings of impending failures. Work signals from strongest to weakest.</li>
<li><strong><em>The Third Way:</em></strong> Use the efficiencies of <strong><em>The First Way</em></strong> and the safety net of <strong><em>The Second Way</em></strong> to introduce rapid experiments that could provide significant gains. Be willing to try 30 ideas to find 1 that succeeds. Make those 30 ideas tiny to keep the feedback cycle fast.</li>
</ol>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>"Developers" get suprisingly little attention and treatment for a book about "dev-ops" but to be fair, the story makes its points without them. <em>Of course</em> there are developers out there that have the completely brazen and careless attitudes depicted in <strong>The Phoenix Project</strong>, but even the smallest dose of nuance would point out that "developer" is not a synonym for "careless cowboy". It was only 2-3 offhand remarks from the protagonist, but clearly it left an impression on me. <a href="https://adamtuttle.codes/blog/2021/thoughts-on-the-phoenix-project/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>!!!!!!!!!!!!!!! <a href="https://adamtuttle.codes/blog/2021/thoughts-on-the-phoenix-project/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
How to Add a 3rd Party Script Tag to a SvelteKit Page Body2021-11-28T00:00:00Zhttps://adamtuttle.codes/blog/2021/adding-3rd-party-script-tags-in-sveltekit/<p><img src="https://adamtuttle.codes/img/2021/danny-howe-bn-D2bCvpik-unsplash.jpg" alt="A giant crowd at a concert, at night" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@dannyhowe?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Danny Howe</a> on <a href="https://unsplash.com/s/photos/party?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>Early in the process of playing with the idea of using SvelteKit for <a href="https://adamtuttle.codes/blog/2021/a-bunch-of-changes/">my website redesign</a> I ran into a couple of issues that I couldn't figure out. I spent a few hours brainstorming, and asking for help in the (<em>excellent!</em>) <a href="https://discord.gg/svelte">Svelte Discord</a>, but wasn't able to come up with a solution.</p>
<p>The biggest problem I had was embedding a 3rd party <code><script></code> tag in the page content.</p>
<p>You might be asking yourself why anyone would do that in 2021. The answer is integrations. The <code><script></code> tag is for the email signup form <a href="https://adamtuttle.codes/#let's-stay-in-touch">on the root page of my site</a>. I don't control its contents (nor do I want to) and it outputs additional HTML at the DOM location where I've included the script tag (think <code>documemt.write</code>)...</p>
<h2 id="embedding-a-script-tag-in-page-content" tabindex="-1">Embedding a Script Tag in Page Content</h2>
<p>If you try to add a <code><script></code> tag to a SvelteKit page to embed an external script, you're probably going to run into an error:</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span><span class="token punctuation">></span></span><span class="token script"><span class="token language-javascript"><br /> <span class="token comment">// normal svelte component stuff...</span><br /></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span><br /><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>h1</span><span class="token punctuation">></span></span>Welcome to the Church of Mountain Dew<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>h1</span><span class="token punctuation">></span></span><br /><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>https://adam-tuttle.ck.page/02c5dc9bec/index.js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script"></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span></code></pre>
<p><img src="https://adamtuttle.codes/img/2021/sveltekit_script_embed_error.png" alt="SvelteKit error message reading, "A component can only have one instance-level element"" /></p>
<p class="photo-byline"><strong>SvelteKit Error:</strong> A component can only have one instance-level <code><script></code> element</p>
<p>So what's the magic fix? It's almost too easy. It came to me totally at random one evening while watching TV with my wife.</p>
<p>Wrap it in an <a href="https://kit.svelte.dev/docs#modules-$app-env"><code>{#if browser}{/if}</code></a> conditional:</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span><span class="token punctuation">></span></span><span class="token script"><span class="token language-javascript"><br /> <span class="token keyword">import</span> <span class="token punctuation">{</span> browser <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'$app/env'</span><span class="token punctuation">;</span><br /></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span><br /><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>h1</span><span class="token punctuation">></span></span>Welcome to the Church of Mountain Dew<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>h1</span><span class="token punctuation">></span></span><br /><br />{#if browser}<br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>https://adam-tuttle.ck.page/02c5dc9bec/index.js<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script"></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span><br />{/if}</code></pre>
<p>No more error, and the script tag is embedded and executes exactly as expected.</p>
<p>Like so many other things in Svelte, when I saw that this was the solution I sat there slack-jawed and asked myself, "Is that really all that's needed?"</p>
<h2 id="alternatives-that-i-considered%2Ftried" tabindex="-1">Alternatives That I Considered/Tried</h2>
<p>You might be wondering why I didn't use <code><svelte:head></code>.</p>
<p>I needed this method specifically because the <code><script></code> tag that I need to embed creates HTML content at the DOM location where the tag exists. In this case, it creates the email list signup form.</p>
<p>If I put my script tag in the document head, the email list signup form might not show up at all, or it might appear in a weird location.</p>
<p>Another alternative would be to use an <code>onMount</code> hook to add it at runtime using something like <code>document.body.appendChild</code>. This would effectively be the same thing as what I've done above with <code>{#if browser}</code>, but less readable and, in my opinion, less expressive. It would also take much more code.</p>
<p>While I wouldn't have been happy with this solution, I did also try to copy the code of the import and try my hand at adding it directly to the page, modifying as necessary to make it work. In this case it just wasn't a workable solution. There's too much code, and it's too complex, to make that a path worth pursuing. Especially once I found the solution described above!</p>
<p>While a need like this is pretty infrequent these days, I'm glad to have figured out how to pull it off with SvelteKit.</p>
<p>Svelte has become a new obsession of mine. You should <a href="https://twitter.com/adamtuttle">follow me on Twitter</a> or <a href="https://adamtuttle.codes/feed/feed.xml">subscribe via rss</a> to follow along as I learn more and share what I learn.</p>
A Bunch of Meta Changes for This Website That Probably Nobody Cares About2021-11-19T00:00:00Zhttps://adamtuttle.codes/blog/2021/a-bunch-of-changes/<p><img src="https://adamtuttle.codes/img/2021/ian-schneider-TamMbr4okv4-unsplash.jpg" alt="" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@goian?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Ian Schneider</a> on <a href="https://unsplash.com/s/photos/change?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>Frequent visitors to this website (so, me) might notice that there are some differences around here. That's because I've spent a bunch of little bits of free time over the last few months working on a bit of a redesign. If for no other reason than to mark the occasion, I thought I would discuss the changes here.</p>
<h2 id="dark-mode!" tabindex="-1">Dark Mode!</h2>
<p>I have been slowly converting my entire life over to full-time darkmode, and I gotta say, it's nice. I think it's harder to make text clean and legible on a dark background than it is on a light background, but it can be done. Hopefully I've succeeded. Plus it was a fun little challenge to learn something new in CSS.</p>
<p>If you toggle your system dark mode setting with this website open, the colors are responsive to the change. You're welcome?</p>
<h2 id="still-on-eleventy-(for-now)" tabindex="-1">Still on Eleventy (for now)</h2>
<p>As with the previous iteration, my blog (still) runs on <a href="https://www.11ty.dev/">Eleventy</a>. And this was exactly where I wanted to be... until a week ago, when I dove into the <a href="https://svelte.dev/">Svelte</a> deep end. It hurts when you're nearly done with a rewrite and realize you want to start from scratch again, but well, here we are.</p>
<p>I did poke at it for one evening and I think it'll be a fun project, but it wasn't easy enough to get off the ground (with the customizations I'll need) to skip the updated Eleventy version, so I'll stick with what I've got for now and maybe run Svelte later.</p>
<h2 id="now-a-digital-garden%2C-no-longer-a-blog" tabindex="-1">Now a Digital Garden, No Longer a Blog</h2>
<p>I don't know that most of you will notice anything different for this one. The TL;DR is that a <a href="https://maggieappleton.com/garden-history">digital garden</a> is intended to be pruned and maintained over time, where a blog is kind of an append-only log. Its information isn't necessarily organized in a chronological way. Check out <a href="https://adamtuttle.codes/">the new home page</a> to see what I mean by this. Thinking about it this way makes me feel freer to go back and delete or change older entries if/when I deem fit; though I don't know how much I'll actually do that.</p>
<p>For SEO reasons I'm keeping the <code>/blog</code> in the URL (easier than fighting redirects and hoping for the best).</p>
<h2 id="so-long-disqus%2C-hello-webmentions" tabindex="-1">So Long Disqus, Hello Webmentions</h2>
<p>This is probably the most impactful of all of the changes. I've been blogging for a long dagum time. Long enough to remember when getting to Google Reader Zero every day was just as important, if not more important than, getting to Inbox Zero (for some of us). Back in those days, before there was Twitter... heck, before there was Facebook, commenting on blogs was <em>so</em> much more common. I think Twitter and Facebook killed blog comments; at least in the communities I frequent.</p>
<p>I'm not going to show you my Disqus stats from the last couple of years, because nobody likes having spiderwebs shoved into their eyeballs. I had ONE post in the last year that garnered more than a single comment, and while there were a couple of single-comment posts, they were outnumbered by no-comments posts by at least 5:1.</p>
<p>And that's fine. The web evolves. That's kind of the only thing it does. Discussions happen vigorously and rapidly on Twitter. Might as well take the discussion to where the people are already talking, right?</p>
<p>I know that some corporate firewalls block Disqus for some reason. Also, I think there are a lot of people that view Disqus as privacy-invading and so they block it or ignore it. That's fair.</p>
<p>Enter <a href="https://webmention.io/">Webmentions</a>. Old-school bloggers may be familiar with the idea of "pingbacks" and Webmentions are sort of a reimagining of that. The short version is that if I tweet a link to a post here, and you reply, your reply will show up as a comment. Also, if you like the tweet it'll show up here as a like. Retweets too, though I've not seen what those look like yet.</p>
<p>I was first exposed to Webmentions by <a href="https://www.swyx.io/clientside-webmentions/">swyx's post on doing it all client-side</a>. I did some initial research at the time but couldn't wrap my head around it. I filed it away to come back to later, and earlier this week I heard about it on a podcast, found <a href="https://sia.codes/posts/webmentions-eleventy-in-depth/">a good guide for my stack</a> (Eleventy, GitHub Actions, and Netlify), and was able to put it together.</p>
<p>I have a reasonably good idea that my posts were getting <em>some</em> engagement on Twitter that wasn't reflected in the old comments, so hopefully this makes it easier to participate, which should hopefully encourage some participation.</p>
<p>To the best of my understanding, webmentions are scraped from Twitter every 30 minutes or so, and I'm running builds of my site on a schedule every half hour now, which should mean that your likes/comments/etc would show up within an hour at the latest. If my math is correct.</p>
<h2 id="contributions-welcome" tabindex="-1">Contributions Welcome</h2>
<p>At the bottom of most pages you'll find links to edit them directly on GitHub. I welcome anyone and everyone to <a href="https://github.com/atuttle/blog/">submit pull requests</a>. If you're making simple text changes like fixing a typo then I find editing directly on GitHub instead of cloning to local to be a perfectly suitable approach, and they make submitting the PR really easy. I intend to add <a href="https://allcontributors.org/">all-contributors-bot</a> to the blog repo, because I want to be transparent about my gratitude for your help.</p>
<p>And if you've never made a pull request before, a copy edit to a blog post is a great way to get your feet wet!</p>
<h2 id="f%C3%ADn." tabindex="-1">Fín.</h2>
<p>That's all for now. If you see anything broken or otherwise messed up, please either let me know or if you're feeling generous and kind you could always <a href="https://github.com/atuttle/blog/">submit a pull request to fix it for me</a>.</p>
Debugging Docker Health Checks2021-11-18T00:00:00Zhttps://adamtuttle.codes/blog/2021/debugging-docker-health-checks/<p><img src="https://adamtuttle.codes/img/2021/barrett-ward-5WQJ_ejZ7y8-unsplash.jpg" alt="Shipping containers in a busy port" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@barrettward?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Barrett Ward</a> on <a href="https://unsplash.com/s/photos/container?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>For what I hope are obvious reasons (deploys, fail-over, etc), docker health checks are important.</p>
<p>Unfortunately they're also a bit of a black box. You write some code to report back whether or not it's healthy, and docker will call that code to determine if the container is healthy, but it's not obvious how to see the results of that code. When a container is being reported as unhealthy, it can be maddening to try and figure out why. Here are some techniques I've found over the years to make debugging them so much less frustrating.</p>
<h2 id="debugging-local-containers" tabindex="-1">Debugging Local Containers</h2>
<p>What is the current health status of a container?</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">docker</span> inspect <span class="token parameter variable">--format</span> <span class="token string">"{{json .State.Health.Status }}"</span> container_name</code></pre>
<p>Let's assume your container is unhealthy. How can you get more detail than that? Firstly, since we're working with JSON on the CLI, I recommend you install <a href="https://stedolan.github.io/jq/">jq</a> (osx homebrew users can run <code>brew install jq</code>).</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">docker</span> inspect <span class="token parameter variable">--format</span> <span class="token string">"{{json .State.Health }}"</span> container_name <span class="token operator">|</span> jq</code></pre>
<p>Just in case you were thinking about not using <code>jq</code>, here's the difference in output of the previous command, without and then with jq:</p>
<p><img src="https://adamtuttle.codes/img/2021/docker-healthcheck-debug-1.png" alt="Terminal screen shot showing the difference between the two commands" /></p>
<h2 id="in-the-cloud" tabindex="-1">In The Cloud</h2>
<p>The only cloud service I have experience with is AWS, so that's the only one I can provide this advice for. Feel free to share the equivalent for Google Cloud, Azure, etc.</p>
<p>Unfortunately AWS doesn't expose the full healthcheck log through their CLI, but you can at least get some sense for it using this command.</p>
<pre class="language-bash"><code class="language-bash">aws ecs describe-tasks <span class="token parameter variable">--cluster</span> <span class="token operator"><</span>cluster-name<span class="token operator">></span> <span class="token parameter variable">--tasks</span> <span class="token operator"><</span>task-id<span class="token operator">></span> <span class="token operator">|</span> jq <span class="token string">'.tasks[0].containers[0]'</span></code></pre>
<p>Of course this requires you to know the task id (at present, this is a 32 character alphanumeric string), which I imagine you can extract using other aws cli commands if needed -- that's out of scope for this discussion.</p>
<p>This returns some useful information but nowhere near the same level of detail as we got for local containers.</p>
<p><img src="https://adamtuttle.codes/img/2021/docker-healthcheck-debug-2.png" alt="Terminal screen shot showing output of the aws ecs describe-tasks call" /></p>
<p>To be perfectly honest, I'm sure most of what I obscured in the above screen shot is benign, but as it's from our production environment and I'm not in a risk-taking mood, better safe than sorry, right?</p>
In Defense of Tiny Modules2021-11-17T00:00:00Zhttps://adamtuttle.codes/blog/2021/in-defense-of-tiny-modules/<p><img src="https://adamtuttle.codes/img/2021/kevin-borrill-IEGWHoS2wY4-unsplash.jpg" alt="Tiny umbrellas at the beach" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@kev2480?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Kevin Borrill</a> on <a href="https://unsplash.com/s/photos/tiny?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>It's popular right now to vehemently hate the proliferation of tiny npm utility modules. But is that a good thing? I don't think so. <a href="https://blog.sindresorhus.com/small-focused-modules-9238d977a92a">And I'm far from the first person to suggest that</a>.</p>
<blockquote>
<p>Imagine if PC manufacturers all made their own CPUs. Most would do it badly. The computer would be more expensive and we would have slower innovation. Instead most use Intel, ARM, etc.</p>
</blockquote>
<p>Here's someone offering to <a href="https://drewdevault.com/2021/11/16/Cash-for-leftpad.html">pay people to delete their (small) modules</a>.</p>
<p>I agree with the basic premise that there's too much garbage on NPM and as a rule of thumb the community at large is too dependent on modules that add yet more dependencies (and so on, and so on) for little value. Nobody likes installing one or two dependencies and then getting 20 vulnerability alerts; most of which are usually ignorable for one reason or another.</p>
<p>This vehement hatred, and the humorous-if-not-satirical offer to pay for deletions, is an overcorrection.</p>
<p>There are problems, but the solution isn't to burn it all down.</p>
<h2 id="why-tiny-modules-are-good" tabindex="-1">Why tiny modules are good</h2>
<p>The article I linked above specifically picks on <a href="https://www.npmjs.com/package/isarray">isArray</a>. And yes, it's true that the code for isArray is just a couple of lines:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">var</span> toString <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">.</span>toString<span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line">module<span class="token punctuation">.</span>exports <span class="token operator">=</span> Array<span class="token punctuation">.</span>isArray <span class="token operator">||</span> <span class="token keyword">function</span> <span class="token punctuation">(</span><span class="token parameter">arr</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">toString</span><span class="token punctuation">.</span><span class="token function">call</span><span class="token punctuation">(</span>arr<span class="token punctuation">)</span> <span class="token operator">===</span> <span class="token string">'[object Array]'</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>There are some subtleties of this code that are worth pointing out:</p>
<ul>
<li>It has tests.</li>
<li>It's tested against every major desktop and mobile browser, and some smaller browsers too.</li>
<li>It's a polyfill. If the execution environment (node, browser) already has <code>Array.isArray</code> defined, it uses that.</li>
</ul>
<p>Would your slapdash from-memory implementation be able to claim the same?</p>
<p>The implementation is <em>just</em> complicated enough that I wouldn't trust myself to remember how to re-implement it from scratch every time I want to use it; which makes it extremely likely that I'm going to find some random function on Stack Overflow and copy/paste it into a utility functions file in my app, and copy/paste that forward into future projects, forever, into oblivion.</p>
<p>This is how we did things before npm, and we stopped doing that because the new way is better.</p>
<p>Sharing the module provides a single, updatable source of truth if a bug or a security vulnerability is found. Contrast that with finding a bug in your utility functions file that you've been copying from project to project for 15 years. In that situation I'm probably not even going to consider going back and cleaning it up in those other projects.</p>
<h2 id="not-invented-here" tabindex="-1">Not Invented Here</h2>
<p>This phenonmenon has a name. <a href="https://en.wikipedia.org/wiki/Not_invented_here">Not Invented Here</a>. If you've been coding on the web long enough, you've run into people who opposed the idea of using a framework for organizing their code, usually with an argument that distilled down to "I don't know how the framework works, so I can't trust it." Instead they prefer to write spaghetti-code or cobble together a system of utility functions for abstracting away repetitive actions; not realizing that what they've done is to reinvent the concept of a framework but to forego the free community testing, proving, bug-hunting, and bug fixes that come from using an open source framework.</p>
<p>The "it's just a couple of lines of code, why install something for it?" argument rings awfully familiar to me.</p>
<h2 id="alternative-solutions" tabindex="-1">Alternative Solutions</h2>
<p>As I said early on, this ecosystem is not without its problems. I just don't think this is one of them.</p>
<p>If you find node_modules expansion particularly egregious, you should seek out and use tiny utility modules only if they have zero dependencies. Dev-dependencies are acceptable, because they don't get installed when you <code>npm install</code>, even if your NODE_ENV is development. Even better, knowing that your utility module library of choice is well tested is good peace of mind. Might I recommend <a href="https://github.com/angus-c/just">just</a>?</p>
<p>Aside from gross file count and size of node_modules, the next most common argument I hear is that vulnerability warnings are too common and usually over-blown. It's a shame that a tool with so much potential for doing good is so noisy that we've all trained ourselves to ignore it, myself included. Npm should notify the module authors and contributors at least as often as it notifies the module users. And there should be a way for vulnerabilities in their database to be marked as "only a problem in production" so that if you're using a vulnerable module downstream of something in your dev-dependencies, it doesn't even bother notifying you. And of course, if something truly malicious were to be found, like crypto-miners running inside your testing framework, then that would raise enough concern to <strong>not</strong> mark it as a "production-only vulnerability."</p>
<p>It doesn't even seem all that far-off. They're already not installing dev-dependencies for modules you install; the threshold for whether or not to display a vulnerability should be nearby, right?</p>
<p>These two changes alone would go a long way to healing the damage done. There's no doubt other things we can do to improve the developer experience of using tiny modules.</p>
<p>I think that the reason it's so popular to hate on tiny modules right now is that (1) the public can't force npm to make these changes, and so (2) it's easier to browbeat others into believing that tiny modules are bad, because it serves your purposes.</p>
<p>Nobody's forcing you to use <em>any</em> modules. If you're upset that a library you want to use relies on modules that rely on modules that rely on modules you don't like, that's a decision you need to make for yourself... But it doesn't make any of the modules in that chain <em>bad</em> or <em>wrong</em>.</p>
<p>If enough people agree with you, then there's a market for you to create a fork of the library that uses few/no dependencies.</p>
SvelteKit: Customizing app.html at Runtime2021-11-16T00:00:00Zhttps://adamtuttle.codes/blog/2021/sveltekit-customizing-app-html-at-runtime/<p>SvelteKit works by starting with a static <code>app.html</code> file that (as of the time of this writing) contains a <code><div id="svelte"></div></code>, into which your Svelte app is hydrated during server-side rendering (SSR); and the Svelte compiler is also aware of this div (thanks to <code>svelte.config.js</code>) so that it can plan its DOM updates accordingly.</p>
<p>For almost all use-cases, this is fine. But what do you do if it's not going to work for <strong>your</strong> use-case? If there's interest I can cover my specific use case in the future, but for now let's focus less on the <em>why</em> and more on the <em>how</em>.</p>
<p>Let's start with the list of requirements that makes this functionality necessary:</p>
<ul>
<li>The "skin" of the page (the html that comes before and after your Svelte app in the document, including the head tag) must be 100% dynamic, determined and loaded at runtime.</li>
<li>Variations of allowable skins that have been pre-loaded at compile time are not sufficient; you need the ability to modify them and see the updates in real-time, without re-compiling the application.</li>
</ul>
<p>If those are your requirements, then this is the solution for you.</p>
<p>First, create a <code>hooks.js</code> file, and export a <code>handle</code> method. We're going to create a <a href="https://kit.svelte.dev/docs#hooks-handle">handle hook</a>.</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// src/hooks.js</span><br /><span class="token keyword">export</span> <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">handle</span><span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> request<span class="token punctuation">,</span> resolve <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> response <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">resolve</span><span class="token punctuation">(</span>request<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> response<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>As written here, this hook does not deviate from the SvelteKit default behavior. We'll get to that shortly. But first we need to tell SvelteKit about our hook. We do that by editing <code>svelte.config.js</code>:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token comment">// svelte.config.js</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> config <span class="token operator">=</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">kit</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">target</span><span class="token operator">:</span> <span class="token string">'#svelte'</span><span class="token punctuation">,</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token literal-property property">files</span><span class="token operator">:</span> <span class="token punctuation">{</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token literal-property property">hooks</span><span class="token operator">:</span> <span class="token string">'./src/hooks.js'</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">}</span></mark><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token keyword">export</span> <span class="token keyword">default</span> config<span class="token punctuation">;</span></span></code></pre>
<p>This tells SvelteKit to look for our custom hooks implementations in the file we created previously.</p>
<p>Now we can get to the fun part.</p>
<p>Back in our custom handle hook, we want to determine what the REAL skin html should be, and inject it.</p>
<ol>
<li>Determine the values that we feed into the algorithm that provides the skin html as its result</li>
<li>Get the skin html</li>
<li>Parse the skin html into useful chunks</li>
<li>Inject those chunks into app.html</li>
</ol>
<h2 id="part-1%3A-determine-input-args-for-skin-lookup" tabindex="-1">Part 1: Determine Input Args for Skin Lookup</h2>
<p>Let's assume the skin can be determined from the domain name and URL string; because that's what my need is.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token comment">// src/hooks.js</span></span><br /><span class="highlight-line"><span class="token keyword">import</span> getSkin <span class="token keyword">from</span> <span class="token string">'$lib/getSkin'</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">export</span> <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">handle</span><span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> request<span class="token punctuation">,</span> resolve <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">const</span> skinHTML <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">getSkin</span><span class="token punctuation">(</span> request<span class="token punctuation">.</span>host<span class="token punctuation">,</span> request<span class="token punctuation">.</span>path <span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token keyword">const</span> response <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">resolve</span><span class="token punctuation">(</span>request<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token keyword">return</span> response<span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<h2 id="part-2%3A-get-the-skin-html" tabindex="-1">Part 2: Get the Skin HTML</h2>
<p>For the sake of clean code and a short-ish article, I'll leave the actual skin html lookup code as an exercise for the reader. In my case, I'm making an HTTP request to an API and the skin html is returned in a property in the JSON response.</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// src/lib/getSkin.js</span><br /><span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">getSkin</span><span class="token punctuation">(</span><span class="token parameter">host<span class="token punctuation">,</span> path</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">const</span> apiRespose <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">fetch</span><span class="token punctuation">(</span><span class="token operator">...</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<h2 id="part-3%3A-parse-the-skin-html" tabindex="-1">Part 3: Parse the Skin HTML</h2>
<p>In order to be easily injected into app.html, we need to do a tiny amount of processing on the HTML we got from the previous step. Ultimately, we want <code>getSkin()</code> to return an object with 3 properties: <code>before</code>, <code>after</code>, and <code>head</code>. The <code>head</code> property will contain everything between the <code><head></head></code> tags in the skin HTML. The <code>before</code> and <code>after</code> properties will contain everything in the <code><body></body></code> tag, but split into two chunks based on a predetermined token. In my case, the skin is required to have the token <code>{{app}}</code>, and so I want <code>before</code> to be everything before <code>{{app}}</code> and <code>after</code> to be everything after it -- from within the <code><body></body></code> tags.</p>
<p>So if my skin HTML looked like this:</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>html</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>head</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>title</span><span class="token punctuation">></span></span>This is my skin<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>title</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>link</span> <span class="token attr-name">rel</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>stylesheet<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/assets/bootstrap.min.css<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>head</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>body</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>div</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>container<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>p</span><span class="token punctuation">></span></span>There are many like it.<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>p</span><span class="token punctuation">></span></span><br /> {{app}}<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>p</span><span class="token punctuation">></span></span>But this one is mine.<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>p</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>div</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>body</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>html</span><span class="token punctuation">></span></span></code></pre>
<p>Then my resulting data response should look like this:</p>
<pre class="language-js"><code class="language-js"><span class="token punctuation">{</span><br /> <span class="token literal-property property">head</span><span class="token operator">:</span> <span class="token string">'<title>This is my skin</title><link rel="stylesheet" href="/assets/bootstrap.min.css" />'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">before</span><span class="token operator">:</span> <span class="token string">'<div class="container"><p>There are many like it.</p>'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">after</span><span class="token operator">:</span> <span class="token string">'<p>But this one is mine.</p></div>'</span><br /><span class="token punctuation">}</span></code></pre>
<p>Note that on their own, the HTML fragments in <code>before</code> and <code>after</code> are malformed HTML (at least for the example skin I've provided). If we tried to accomplish what I'm about to do with the handle hook using <code>{@html before}</code> in a layout, it would only work if the HTML fragments are <em>not</em> malformed.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2021/sveltekit-customizing-app-html-at-runtime/#fn1" id="fnref1">[1]</a></sup> Part of what makes this approach better is that it doesn't suffer from that limitation.</p>
<p>How do you parse these sections out of a single HTML string? Like this:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token comment">// src/lib/getSkin.js</span></span><br /><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">parseSkin</span><span class="token punctuation">(</span>layout<span class="token punctuation">,</span> appToken <span class="token operator">=</span> <span class="token string">'{{app}}'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">let</span> <span class="token punctuation">[</span>before<span class="token punctuation">,</span> after<span class="token punctuation">]</span> <span class="token operator">=</span> layout<span class="token punctuation">.</span><span class="token function">split</span><span class="token punctuation">(</span>appToken<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> <span class="token comment">//linebreaks get in the way of our regexes, so remove them.</span></span><br /><span class="highlight-line"> before <span class="token operator">=</span> before<span class="token punctuation">.</span><span class="token function">replaceAll</span><span class="token punctuation">(</span><span class="token string">'\r'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">replaceAll</span><span class="token punctuation">(</span><span class="token string">'\n'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> after <span class="token operator">=</span> after<span class="token punctuation">.</span><span class="token function">replaceAll</span><span class="token punctuation">(</span><span class="token string">'\r'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">replaceAll</span><span class="token punctuation">(</span><span class="token string">'\n'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> <span class="token comment">//pull <head> stuff out of the skin and drop it in here:</span></span><br /><span class="highlight-line"> <span class="token keyword">let</span> head <span class="token operator">=</span> before<span class="token punctuation">.</span><span class="token function">match</span><span class="token punctuation">(</span><span class="token regex"><span class="token regex-delimiter">/</span><span class="token regex-source language-regex"><head>(.*)<\/head></span><span class="token regex-delimiter">/</span><span class="token regex-flags">gi</span></span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>head<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> head <span class="token operator">=</span> head<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token string">'<head>'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token string">'</head>'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> head <span class="token operator">=</span> <span class="token string">''</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> <span class="token comment">//convert <body whatever="doesn't matter"> to <body></span></span><br /><span class="highlight-line"> before <span class="token operator">=</span> before<span class="token punctuation">.</span><span class="token function">replaceAll</span><span class="token punctuation">(</span><span class="token regex"><span class="token regex-delimiter">/</span><span class="token regex-source language-regex"><body([^>]+)></span><span class="token regex-delimiter">/</span><span class="token regex-flags">gi</span></span><span class="token punctuation">,</span> <span class="token string">'<body>'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> <span class="token comment">//delete everything up to and including <body></span></span><br /><span class="highlight-line"> before <span class="token operator">=</span> before<span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token regex"><span class="token regex-delimiter">/</span><span class="token regex-source language-regex">^.+<body></span><span class="token regex-delimiter">/</span><span class="token regex-flags">i</span></span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> <span class="token comment">//delete everything including and after </body></span></span><br /><span class="highlight-line"> after <span class="token operator">=</span> after<span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token regex"><span class="token regex-delimiter">/</span><span class="token regex-source language-regex"><\/body>.*$</span><span class="token regex-delimiter">/</span></span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> <span class="token keyword">return</span> <span class="token punctuation">{</span> before<span class="token punctuation">,</span> after<span class="token punctuation">,</span> head <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">getSkin</span><span class="token punctuation">(</span><span class="token parameter">host<span class="token punctuation">,</span> path</span><span class="token punctuation">)</span><span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">const</span> apiRespose <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">fetch</span><span class="token punctuation">(</span><span class="token operator">...</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">return</span> <span class="token function">parseSkin</span><span class="token punctuation">(</span> apiResponse<span class="token punctuation">.</span>skinHTML <span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<h2 id="part-4%3A-inject-skin-parts-into-app.html" tabindex="-1">Part 4: Inject Skin Parts Into app.html</h2>
<p>Now that we have head/before/after, we need to modify the response to inject our HTML fragments in the correct locations. There's a few different ways you could do this, but here's what I did.</p>
<p>I started by modifying app.html to add some tokens indicating locations for my additions:</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line"><span class="token doctype"><span class="token punctuation"><!</span><span class="token doctype-tag">DOCTYPE</span> <span class="token name">html</span><span class="token punctuation">></span></span></span><br /><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>html</span> <span class="token attr-name">lang</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>en<span class="token punctuation">"</span></span><span class="token punctuation">></span></span></span><br /><span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>head</span><span class="token punctuation">></span></span></span><br /><span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>meta</span> <span class="token attr-name">charset</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>utf-8<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span></span><br /><span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>link</span> <span class="token attr-name">rel</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>icon<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/favicon.png<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span></span><br /><span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>meta</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>viewport<span class="token punctuation">"</span></span> <span class="token attr-name">content</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>width=device-width, initial-scale=1<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span></span><br /><mark class="highlight-line highlight-line-active"> %tpl.head%</mark><br /><span class="highlight-line"> %svelte.head%</span><br /><span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>head</span><span class="token punctuation">></span></span></span><br /><span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>body</span><span class="token punctuation">></span></span></span><br /><mark class="highlight-line highlight-line-active"> %tpl.before%</mark><br /><span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>div</span> <span class="token attr-name">id</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>svelte<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>%svelte.body%<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>div</span><span class="token punctuation">></span></span></span><br /><mark class="highlight-line highlight-line-active"> %tpl.after%</mark><br /><span class="highlight-line"> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>body</span><span class="token punctuation">></span></span></span><br /><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>html</span><span class="token punctuation">></span></span></span></code></pre>
<p>The <code>%svelte.head%</code> token was already there and is used by SvelteKit to inject the content you specify with <code><svelte:head></code>; so I built on that format to add my tokens. You could instead do the same with HTML comments, if you like that approach better: <code><!-- TPL.HEAD --></code>. It's just a string that will appear in the response buffer after SvelteKit does its SSR that you're going to replace – so do make it pretty unique and unlikely to appear in the app body.</p>
<p>Now doing the string replace becomes trivial:</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// src/hooks.js</span><br /><span class="token keyword">import</span> getSkin <span class="token keyword">from</span> <span class="token string">'$lib/getSkin'</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">export</span> <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">handle</span><span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> request<span class="token punctuation">,</span> resolve <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> response <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">resolve</span><span class="token punctuation">(</span>request<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">const</span> <span class="token punctuation">{</span> head<span class="token punctuation">,</span> before<span class="token punctuation">,</span> after <span class="token punctuation">}</span> <span class="token operator">=</span> <span class="token function">getSkin</span><span class="token punctuation">(</span>request<span class="token punctuation">.</span>host<span class="token punctuation">,</span> request<span class="token punctuation">.</span>path<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> response<span class="token punctuation">.</span>body <span class="token operator">=</span> response<span class="token punctuation">.</span>body<span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token string">'%tpl.head%'</span><span class="token punctuation">,</span> head<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> response<span class="token punctuation">.</span>body <span class="token operator">=</span> response<span class="token punctuation">.</span>body<span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token string">'%tpl.before%'</span><span class="token punctuation">,</span> before<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> response<span class="token punctuation">.</span>body <span class="token operator">=</span> response<span class="token punctuation">.</span>body<span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token string">'%tpl.after%'</span><span class="token punctuation">,</span> after<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> response<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>And lastly, for performance reasons and to not interfere with requests for assets other than HTML pages, we can add a simple conditional:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token comment">// src/hooks.js</span></span><br /><span class="highlight-line"><span class="token keyword">import</span> getSkin <span class="token keyword">from</span> <span class="token string">'$lib/getSkin'</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">export</span> <span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">handle</span><span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> request<span class="token punctuation">,</span> resolve <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">const</span> response <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">resolve</span><span class="token punctuation">(</span>request<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">if</span> <span class="token punctuation">(</span>response<span class="token punctuation">.</span>headers<span class="token punctuation">[</span><span class="token string">'content-type'</span><span class="token punctuation">]</span> <span class="token operator">===</span> <span class="token string">'text/html'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></mark><br /><span class="highlight-line"> <span class="token keyword">const</span> <span class="token punctuation">{</span> head<span class="token punctuation">,</span> before<span class="token punctuation">,</span> after <span class="token punctuation">}</span> <span class="token operator">=</span> <span class="token function">getSkin</span><span class="token punctuation">(</span>request<span class="token punctuation">.</span>host<span class="token punctuation">,</span> request<span class="token punctuation">.</span>path<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> response<span class="token punctuation">.</span>body <span class="token operator">=</span> response<span class="token punctuation">.</span>body<span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token string">'%tpl.head%'</span><span class="token punctuation">,</span> head<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> response<span class="token punctuation">.</span>body <span class="token operator">=</span> response<span class="token punctuation">.</span>body<span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token string">'%tpl.before%'</span><span class="token punctuation">,</span> before<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> response<span class="token punctuation">.</span>body <span class="token operator">=</span> response<span class="token punctuation">.</span>body<span class="token punctuation">.</span><span class="token function">replace</span><span class="token punctuation">(</span><span class="token string">'%tpl.after%'</span><span class="token punctuation">,</span> after<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"> <span class="token keyword">return</span> response<span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>From a technical standpoint, that's pretty much it. Of course you'll want to cache your skin HTML lookup for whatever amount of time makes sense for you for performance reasons, but short of that and the api result fetch, this is everything you need.</p>
<p>While most of the code here is of my own design, I should note that the approach used in the handle hook itself was <a href="https://github.com/sveltejs/kit/pull/2773#issuecomment-967099525">suggested to me by Rich Harris</a>, the creator of Svelte, when I had proposed a different way of accomplishing the same goal -- because this way is much more germane to the project. Indeed, it requires no modification to SvelteKit at all.</p>
<p>I also got lots of additional help along the way from various helpful people in the <a href="https://svelte.dev/chat">Svelte Discord</a>, and I'm happy to report that the people there have always been very kind and generous in helping me figure out what I'm doing wrong. But it's fair to say that the <em>best</em> help I got came only after I put in the effort to <a href="https://github.com/sveltejs/kit/pull/2773">make a pull request with my own idea</a>; pretty clearly illustrating the power of <a href="https://meta.wikimedia.org/wiki/Cunningham%27s_Law">Cunningham's Law</a>.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>I know, because I tried that first. <a href="https://adamtuttle.codes/blog/2021/sveltekit-customizing-app-html-at-runtime/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Is TDD Right For Me?2021-10-27T00:00:00Zhttps://adamtuttle.codes/blog/2021/is-tdd-right-for-me/<p><img src="https://adamtuttle.codes/img/2021/joshua-j-cotten-7ZppYEoBNhQ-unsplash.jpg" alt="A confused bird on a hummingbird feeder" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@jcotten?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Joshua J. Cotten</a> on <a href="https://unsplash.com/s/photos/unsure?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>I believe in automated testing.</p>
<p>I believe that automated testing is table-stakes for professional software development of any kind – I just so happen to focus on the web these days.</p>
<p>I believe that if you don't have decent automated tests for your code, you're behind and playing catch-up, or at worst, waiting for a piano to drop on you from above. <em>This applies to most of my code, so if I'm calling anyone out, it's me.</em></p>
<p>I am <a href="https://adamtuttle.codes/blog/2021/tdd-by-example-kent-beck/">in the process of reading</a> what I understand to be the TDD bible: "TDD By Example" by Kent Beck. (Yeah, I'm way behind schedule. Sorry.)</p>
<p>Amongst the many things I do every week, testing seems to come up with extreme regularity. If I'm not discussing it on <a href="https://workingcode.dev/">my podcast</a> then I'm writing tests at work and focusing on honing that skill or —most often— discussing aspects of it in <a href="https://workingcode.dev/discord">the podcast discord</a>. There's a certain person there who fancies himself competent at TDD, and certainly he's better than me, so I try to learn what I can from him by sending him my random thoughts and questions to see what he makes of them.</p>
<p>I'm pretty sure that's what inspired his post <a href="https://blog.adamcameron.me/2021/04/tdd-is-not-testing-strategy.html">TDD is not a testing strategy</a>, in which he frustratedly explains TDD is a method for <em>designing software</em>, not for <em>writing tests</em>.</p>
<p>This is difficult for me because I have been writing software professionally (getting paid for it) since roughly 1998-1999. I'll do the math for you: that's well over 20 years. <strong>And while I had heard of automated testing occasionally in my career, not a single employer, project manager, team lead, or mentor along the way stressed the importance of TDD or even automated testing for that matter.</strong> Which brings me to today.</p>
<p>Please allow me a moment of hubris: I am reasonably f***n' good at my job.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2021/is-tdd-right-for-me/#fn1" id="fnref1">[1]</a></sup></p>
<p>An average project for me in the last 10 years consists of a simple request for a feature addition, perhaps as compared to an existing feature in a different part of our product but different in some way, and most often a request for me to determine the amount of time I need to complete it. Not often, but sometimes the deadline is pre-determined, and I've become quite adept at the conversation about which features need to be kept and which can get tossed overboard at the last minute in order to ship on time if I can't get them done. That's a different topic altogether.</p>
<p>I specifically wanted to point out that I often get to dictate the timeline, because that's so often an excuse for lack of testing. I don't have that excuse.<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2021/is-tdd-right-for-me/#fn2" id="fnref2">[2]</a></sup></p>
<p>Because I have 20 years of experience in developing software from minimal specifications and meeting management and customer expectations, and having done so without using TDD as a method to <em>design</em> that software, I have developed an ability to near-instantly start to understand how I will break the problem down and implement it (in broad strokes, at least) the moment it is asked of me.</p>
<p>90% of web development is accepting inputs (form, url), reading/writing against a database, and spitting some data out on a resulting page. That doesn't take a rocket surgery degree. It's not hard to predict where the breakpoints between view, controller, service, and DAO will be; nor what sorts of tasks will be done in each spot. All that's left to do is:</p>
<ul>
<li>Name the methods</li>
<li>Define the method signatures (what arguments do they take and what do they return?)</li>
<li>Implement the body of those methods, recursing as necessary to refactor out complex or reusable bits of code</li>
<li>Display the results in a way that's meaningful to the humans that will be using it</li>
</ul>
<p>That sounds an awful lot like what I'd expect to see in a specification document, if I ever got one of those. You might think I should take a few moments to put those thoughts down in writing before I begin, and boom, there's your spec. But therein lies the rub.</p>
<p>I always dive in and start coding with some assumptions about what the method signature should be (what I would have written down in the spec), and often as I'm writing the implementation I realize that I hadn't thought about certain parts of the problem fully, and as a result I need to add another argument, or change the type of an existing argument. I traverse up and down the stack from view to controller to service and DAO and back again, continuously, rapidly evolving the functionality, method signatures, view contents, etc. Usually my physical ability to navigate the code and type the changes is the limiting factor — I can think of the changes a heck of a lot faster than I can make them.</p>
<p>The code often evolves so much during this process that it barely resembles the original (mental) specification at all by the end.</p>
<p>Maybe this sounds awful to you. That's fine. I'm not here to yuck your yum or to claim that TDD is somehow bad or wrong.</p>
<p>But you know what? This process works <em>fantastically</em> for me. I hope that nobody would be so bold to claim that "TDD is the only correct way to design software," but if they did I would disagree. I think that if the goal is to take certain inputs ("we need this feature, it should be kinda like that other feature, but different in XYZ way..."), and produce software that satisfies that need, then there's multiple ways to get there.</p>
<p>That tests are a byproduct of TDD would be nice, but if it's faster and easier for me to add them after the fact, I don't think that somehow makes my approach "wrong."</p>
<p>Anyway, I'm still going to <a href="https://adamtuttle.codes/blog/2021/tdd-by-example-kent-beck/">finish reading TDD By Example</a><sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2021/is-tdd-right-for-me/#fn3" id="fnref3">[3]</a></sup>, and I am completely open to having my mind changed. It's just that I am starting to realize that TDD may not be a universally "best" design process. In some ways, this is my way of giving myself permission to choose not to follow TDD after I finish the book.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>Except for that whole not much testing thing... <a href="https://adamtuttle.codes/blog/2021/is-tdd-right-for-me/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>Purists would argue that nobody has that excuse because the tests should be the absolute last thing you throw overboard to meet the deadline, and if you don't have time to test then you don't have time to write the feature... But again, I'm not making that argument. I mostly set my own timelines. <a href="https://adamtuttle.codes/blog/2021/is-tdd-right-for-me/#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p>Some day, presumably. <a href="https://adamtuttle.codes/blog/2021/is-tdd-right-for-me/#fnref3" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
What I Wish I Knew for My First Day on PostgreSQL2021-09-20T00:00:00Zhttps://adamtuttle.codes/blog/2021/what-i-wish-i-knew-my-first-day-on-postgres/<p><img src="https://adamtuttle.codes/img/2021/andrew-rice-xYO4F6HoxOQ-unsplash.jpg" alt="An elephant sprays itself with mud" />
<em>Photo by <a href="https://unsplash.com/@andrewricegolf?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Andrew Rice</a> on <a href="https://unsplash.com/s/photos/elephant?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></em></p>
<p>I have only just started using PostgreSQL for a new application; and I'm coming from a MySQL background. That affords me a pretty nifty advantage: <em>The eye of the newbie!</em> So I thought I would share the things that were important to me as I begin my PGSQL journey, in hopes that they can save you some time as you start yours.</p>
<h2 id="choosing-a-gui" tabindex="-1">Choosing a GUI</h2>
<p>There's lots of choices for a GUI. I'm not here to shill for any of them. Since this is all new to me I'm not looking to make any big investments right now so I'm sticking to the free tools. I've heard recommendations for both <a href="https://dbeaver.io/">DBeaver</a> and <a href="https://eggerapps.at/postico/">Postico</a>, and I also stumbled on <a href="https://www.pgadmin.org/">pgAdmin</a>. I started with DBeaver based on a recommendation from a coworker, but so far I'm finding more that I dislike about it than I like.</p>
<h2 id="primary-key-data-type" tabindex="-1">Primary Key Data Type</h2>
<p>Coming from MySQL where the norm is to use the <code>int</code> type with an <code>auto_increment</code> flag for your primary key (which generates auto-incrementing numeric value for you at each new insert), the first real question I had was how to do the same in PG. A quick google told me to use the <code>serial</code> type (there's also <code>big serial</code> if you need it), which does basically the same thing.</p>
<p>While we're on types, here are a few other conversions as I currently understand them:</p>
<ul>
<li><code>varchar(42)</code> 👉 <code>character varying(42)</code></li>
<li><code>char(2)</code> 👉 <code>character(2)</code></li>
</ul>
<h2 id="time-zones" tabindex="-1">Time Zones</h2>
<p>My company's customers are spread across many different time zones, so yes, <a href="https://www.youtube.com/watch?v=-5wpm-gesOY">we get to live in that hell</a>. But we've found some approaches that work well for us, largely depending on storing dates in UTC in the database and using its TZ functions to do conversions as needed. I haven't got this 100% worked out yet but I'm sure that <a href="https://www.postgresql.org/docs/13/datatype-datetime.html#DATATYPE-TIMEZONES">this page of the documentation</a> covers the basics pretty well.</p>
<p>tl;dr: (at least as far as I understand yet) <strong>don't</strong> use <code>time with time zone</code> instead preferring <code>timestamp with time zone</code>. If time zones are at all important to you (and they probably are) then use the <code>... with time zone</code> column types everywhere.</p>
<h2 id="case-sensitivity-is-a-whole-thing" tabindex="-1">Case Sensitivity Is a Whole Thing</h2>
<p>Look, there's work-arounds. But it's a frustrating struggle-bus and imho not worth fighting. Make everything lower case (use <code>snake_case_column_names</code> if you must get that visual separation). For the other benefits like Common Table Expressions ("CTE's") and the JSON support, this is a small price to pay. And it probably hurts me as much as it could hurt anyone, because I've been a lifelong devotee to <code>camelCaseColumnNames</code> and that muscle memory isn't going to rewrite itself easily.</p>
The Mistake that Every TDD Tutorial Makes2021-09-09T00:00:00Zhttps://adamtuttle.codes/blog/2021/the-mistake-every-tdd-tutorial-makes/<p>I refactored this article out of <a href="https://adamtuttle.codes/blog/2021/tdd-by-example-kent-beck/">my notes on the book TDD By Example by Kent Beck</a> because it's my website and I can do what I want. 😜 While what I have to say here sounds negative —even to me— I am actually quite excited to learn TDD and improve my testing skills. I guess I'm just tired of being treated like a "child," as it were. Thanks for indulging me.</p>
<p>I hold the opinion that "automated tests are good" and that there are good ways and bad ways to test things, and that there is significant value in using the correct approach. E.g. test the feature, not the implementation. I also believe that TDD <em>might</em> be a useful tool to ensure that you've written tests to cover every feature. I haven't finished reading the [TDD Bible][tddbyex] yet, so I am withholding my opinion on that for now.</p>
<p>But <em>every</em> TDD tutorial I've encountered to date has gone out of its way to "test" such inane details <em>of the testing-setup itself</em> that I can't help but feel that, as my English friends would say, they're "taking the piss." They're going to the extreme, perhaps to make a point, but also in a manner that is so unbelievably inefficient and annoying as to get in the way of making that point!</p>
<blockquote>
<p><em><strong>If you wish to write software with TDD, you must first invent the universe.</strong></em></p>
</blockquote>
<p>Do you really, for <em>every</em> project, <em><strong>really</strong></em> start by writing a test that won't even <em>compile</em>? And then once you've written enough code to make it compile, writing the first test in <em>such a naïve manner</em> that nobody would actually look at it and think it's actually a useful test?</p>
<p>Of course you don't. Because that would be a waste of your own time, and that's not what TDD is about, is it?</p>
<p><strong>Be honest with your audience.</strong></p>
<p>Tell them that half of the point is to learn how to break things down into the absolute smallest possible testable-chunks, which sometimes are even just a single character change; and the other half of the point is to learn how much is too much without stopping to add more tests. (Did I get that right? I'm no TDD expert.) (Note: Reserving this space for whatever additional/updated thoughts I might have after I finish reading TDD By Example.)</p>
<p>Would you invite Isaac Newton in for a lesson on Quantum Computing and start by asking him to show his work on some long division?</p>
<p><em>God, I hope not.</em></p>
<p>No, you show respect for his established skills and you try to meet him at a place that makes sense given that context to create a pleasant on-ramp into the topic at hand.</p>
<p>I am not comparing myself to Isaac Newton – far from it. But taking examples to their extremes is often illustrative, as I hope this one shows.</p>
<p>My point is that there's a huge difference between teaching someone new to development and teaching someone with 20 years of experience, and those different audiences need different approaches.</p>
<p>A beginner might not have the experience to look at some code and know that it won't compile because the class under test isn't even defined.</p>
<p>The experienced developer is doing that pre-compile step in their head, and there's nothing wrong with that. The step doesn't add any lasting value if it doesn't live on in the tests once the exercise is over. Its only value was in demonstrating just how microscopic a change is testable.</p>
<p>That's it. Maybe consider adding a different introductory chapter/section of your guide specifically for developers with lots of experience coding, and even experience writing tests, who are following your guide specifically to learn the TDD part.</p>
<p>Whatever you do, try to meet your students <em>where they are</em>.</p>
Notes on TDD By Example, by Kent Beck2021-09-06T00:00:00Zhttps://adamtuttle.codes/blog/2021/tdd-by-example-kent-beck/<p>To the best of my knowledge, <a href="https://amzn.to/35hV6X3"><strong>TDD By Example</strong> by Kent Beck</a> is the <em>bible</em> of TDD, and I want to get better at testing and stop pretending that I understand how TDD is different from just making sure you have tests, so I bought a copy. What follows is the notes I took while I was reading it. It's a mixture of the lessons in the book and my thoughts on the book, the lessons, and the teaching methods.</p>
<p><a href="https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530?crid=1D8X147Q3MS5G&dchild=1&keywords=tdd+by+example&qid=1623418032&sprefix=tdd+by+example%2Caps%2C136&sr=8-3&linkCode=li3&tag=tuttl-20&linkId=f4d8afaabf721a86be63f5b8d30c708a&language=en_US&ref_=as_li_ss_il" target="_blank"><img src="https://adamtuttle.codes/img/2021/tdd-by-example-cover.jpg" style="border: 0; max-width: 75%; margin: 0 auto; display: block;" /></a><img src="https://ir-na.amazon-adsystem.com/e/ir?t=tuttl-20&language=en_US&l=li3&o=1&a=0321146530" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /></p>
<h2 id="first-impressions" tabindex="-1">First Impressions</h2>
<p>I am not usually one to read the preface or introduction of books. I don't usually have much in the way of patience. For whatever reason, my ADHD didn't get in the way this time and I chose to read them.</p>
<p>The preface is skippable if you really want to cut corners, but if you think you could benefit from some additional motivation and inspiration, I'd say read it. It's only a couple of pages and does a nice job of setting the mood and purpose of the book, and putting the reader into the right headspace to take on the challenges that lay ahead. It lays out the "red, green, refactor" mantra we've all heard and <em>think</em> we understand, and explains that with proper application, tests provide what the book calls "courage."</p>
<p><strong>The introduction is required reading.</strong> It explains a real problem that a real developer was asked to solve for a real app (<a href="http://c2.com/doc/oopsla92.html">WyCash+</a>) - It managed cash portfolios only in USD, and they wanted to know if it could be improved to be multi-currency. It lays out some of the challenges and how a decent application architecture and appropriate testing gave the developers confidence that they could make the requested changes – it wasn't even a question of how long it would take, it was, "can it be done?" This paragraph does not cover the entire introduction, it's still required reading. ;)</p>
<p>After reading the first chapter, which I'll discuss next, I was astonished by how short it was. Only 8 pages, two of which contain 3/4-page height screen shots of the desktop application GUI for JUnit. I quickly thumbed through the first half of the book and found that every chapter appears to follow pretty much the same format. Just a few pages. I probably should have seen it coming, since TDD is all about breaking things down into the smallest possible useful segments. It was a pleasant surprise, and has contributed greatly to my motivation to keep reading.</p>
<p>There are 36 chapters, if you include the preface and introduction and two appendices – all of which are just as short as the chapters, some even being reduced to just a page or three. If you set out to read one chapter per day —a very attainable goal!— then you would probably find you had finished the entire book in less than a month, because there are likely to be days where you finish 1 chapter and feel eager to keep going.</p>
<p>So that's what I'm going to do. Starting today, I'm planning to read one chapter per day, and I'll take notes about what I read here. I'll be updating this article each time I have more to add.</p>
<p>With the stage sufficiently set, let's jump into the first chapter.</p>
<h2 id="chapter-1%3A-multi-currency-money" tabindex="-1">Chapter 1: Multi-Currency Money</h2>
<p>Unfortunately, <strong>TDD By Example</strong> leads off with the same premise that every other TDD guide does: The premise that the reader is relatively new to coding. <a href="https://adamtuttle.codes/blog/2021/the-mistake-every-tdd-tutorial-makes/">This is a pet-peeve of mine</a>. I've tried on several occasions to learn TDD and been chased away by just how inane and inefficient it's made to look by starting from first principles. <em>If you wish to write software with TDD, you must first invent the universe.</em></p>
<p>Obviously-failing-tests-failing-obviously aside (confused? read my rant on <a href="https://adamtuttle.codes/blog/2021/the-mistake-every-tdd-tutorial-makes/">the mistake that every TDD tutorial makes</a>), I did actually learn something from the first chapter.</p>
<p>In addition to red/green/refactor, you'll also commonly see these as the steps of TDD (copied here from chapter 1):</p>
<ol>
<li>Add a little test</li>
<li>Run all tests and fail</li>
<li>Make a little change</li>
<li>Run the tests and succeed</li>
<li>Refactor to remove duplication</li>
</ol>
<p>Upon first encountering this list, I couldn't fathom what sort of duplication might exist. The whole point of this process is to test so fervently-tiny a component, how could we possibly have duplicated anything?</p>
<p>As it turns out, the duplication in question —at least for the purposes of chapter 1— is between the (naïve) implementation and the (naïve) test. Duplication between the test and the implementation is an indicator that the implementation is incomplete. Something is hard-coded. The book updates the implementation to the following:</p>
<pre class="language-java"><code class="language-java"><span class="token keyword">class</span> <span class="token class-name">Dollar</span> <span class="token punctuation">{</span><br /> <span class="token keyword">int</span> amount<span class="token punctuation">;</span><br /><br /> <span class="token class-name">Dollar</span><span class="token punctuation">(</span><span class="token keyword">int</span> amount<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">this</span><span class="token punctuation">.</span>amount <span class="token operator">=</span> amount<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">void</span> <span class="token function">times</span><span class="token punctuation">(</span><span class="token keyword">int</span> multiplier<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> amount <span class="token operator">*=</span> multiplier<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>At this point the test passes and the implementation code resembles the 124 characters I probably would have started with in the first place before ever running the first test. And when I did write that test, I would, less-naïvely, have included two different math assertions to prove that the tests weren't passing because of any hard-coded values.</p>
<pre class="language-java"><code class="language-java"><span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">testMultiplication</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token class-name">Dollar</span> five <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Dollar</span><span class="token punctuation">(</span><span class="token number">5</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> five<span class="token punctuation">.</span><span class="token function">times</span><span class="token punctuation">(</span><span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">assertEquals</span><span class="token punctuation">(</span><span class="token number">10</span><span class="token punctuation">,</span> five<span class="token punctuation">.</span>amount<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> five<span class="token punctuation">.</span><span class="token function">times</span><span class="token punctuation">(</span><span class="token number">3</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">assertEquals</span><span class="token punctuation">(</span><span class="token number">30</span><span class="token punctuation">,</span> five<span class="token punctuation">.</span>amount<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>This raises the question of immutability, but I'm sure we'll get to that.</p>
<p>I suppose the point of all of this is to illustrate the steps that you should follow <strong>when the problem is one that you can't solve in your head</strong>.</p>
<ol>
<li>First make a test that, when it passes, will prove that your feature works as expected. This is part of what upsets me about the initial test case: only testing 1 set of inputs doesn't fully prove the feature works; only that you are capable of understanding how the test works.</li>
<li>Then iterate on the application to make the test pass, committing whatever sins of <a href="https://workingcode.dev/episodes/022-book-club-1-clean-code-by-uncle-bob-martin-pt1/">"Clean Code"</a> are necessary to get there.</li>
<li>Once you're satisfied that the code actually works and the test actually proves the code works, refactor the code to be "Clean" while still passing the entire suite of tests to prove that you haven't broken anything else in the process.</li>
</ol>
<h2 id="chapter-2%3A-degenerate-objects" tabindex="-1">Chapter 2: Degenerate Objects</h2>
<p>Chapter 2 identifies and resolves another code smell: the immutability concern I raised near the end of my notes on Chapter 1. We're seeing the refactoring process at work, but there's not much going on here. We're still in the neighborhood of obviousness.</p>
<p>But it also validates my complaints about chapter 1 and about TDD guides in general: If there's an obvious implementation, use it!</p>
<blockquote>
<p>"When I use TDD in practice, I commonly shift between these two modes of implementation [Fake it vs. Use Obvious Implementation]. When everything is going smoothly and I know what to type, I put in Obvious Implementation after Obvious Implementation."</p>
</blockquote>
<p>It's nice to see this somewhat early in the book, but considering that the preface and introduction are both strongly recommended reading and both full-chapter-length, this is basically in chapter 4.</p>
<h2 id="chapter-3%3A-equality-for-all" tabindex="-1">Chapter 3: Equality for All</h2>
<p>Here we're starting to see the todo list in action as we add both <code>equals()</code> and <code>hashCode()</code> as items to come back to, and then we start writing a test for the former. I would have sworn there was a note somewhere early on (introduction? preface? chapter 1?) about some helpful tips coming for the todo list, but so far I'm not seeing anything aside from "use one." For lack of a better place, I mentally picture it as a comment at the top of the tests file with a heading of "TODO."</p>
<p>"Triangulation" is presented as a technique of adding multiple assertions to help clarify what the right implementation should be if you're unsure. Sound familiar? That's what I was suggesting in my chapter 1 notes as table-stakes for all tests. I think if you're only doing 1 assertion per test you can't be sure that the code <em>actually works</em> can you? Ah well, at least this was a familiar technique.</p>
<h2 id="chapter-4%3A-privacy" tabindex="-1">Chapter 4: Privacy</h2>
<p>This chapter was about restoring some "cleanliness" to the code and the tests. We made the instance <code>amount</code> variable private, and refactored the tests to be more "expressive." Instead of a series of steps that ends with an assertion, the test we refactored starts to look more like what passes for conversational among coders. (BDD syntax would be even nicer, but that seems out of scope.)</p>
<p>Also this is where we're introduced to the idea that if a bug were to slip past our tests into production, it should be used as an opportunity to learn what test we should have written. What's not expressly said (to the detriment of the book) is that in this scenario you should write the test that shows the code is broken by failing, and then make it pass by fixing the bug.</p>
<h2 id="chapter-5%3A-franc-ly-speaking" tabindex="-1">Chapter 5: Franc-ly Speaking</h2>
<p>Now we've arrived at the part of the story where we have some functionality for dollars and we want to make that same functionality work for Francs. Of course, since this book is all about <s>living out your wildest fantasies</s> committing sins in the name of getting green tests as fast as possible, we simply copy/paste/rename the class and the tests from Dollars to Francs.</p>
<p>From here I've got more questions. Are we creating duplicate tests that will become obsolete and possibly even deleted later on when we refactor the code?</p>
<p>Again, I'm willing to suspend my disbelief for the duration of the book under the premise that we're learning a technique with over-simplified examples. If this were real code we'd start with the obvious inheritence implementation and tests that expect it, right? I guess we'll see.</p>
Taffy 3.3 Release: What's Inside?2021-08-24T00:00:00Zhttps://adamtuttle.codes/blog/2021/taffy-3-3-whats-inside/<p>Taffy 3.2.0 was an extremely stable release, as evidenced by the fact that it was published on November 10th 2017 (nearly 4 years ago!) and that was the last release until today.</p>
<p style="text-align: center"><a href="https://github.com/atuttle/Taffy/releases/tag/v3.3.0" class="btn">Taffy 3.3.0 is out now! Download it here!</a></p>
<p><img src="https://adamtuttle.codes/img/2021/taffy-330-improved-docs.png" alt="Screen shot of Taffy 3.3.0 improvements to the inline documentation in the dashboard" /></p>
<p>To be fair, much of what's new in Taffy 3.3.0 has been there for a while. I was mostly just too busy or too lazy to create an official release for it. But some new features that I've added in recent weeks definitely deserve some attention and a new version number. As (almost) always, you can check the release notes for <a href="https://docs.taffy.io/#/3.3.0?id=what39s-new-in-330">all details of what's new in Taffy 3.3.0</a>, but here are some highlights:</p>
<ul>
<li>No breaking changes! 🎉 Since Taffy obeys <a href="http://semver.org/">SemVer 2.0.0</a> versioning rules, any breaking changes will result in the major version number increasing (e.g. 4.x.x). Thus, you should be able to upgrade from any 3.x.x version with high confidence that nothing will break!</li>
<li>Instantly find the resource you're looking for with a search-as-you-type filter at the top of the dashboard. Thanks to <a href="https://github.com/JamoCA">James Moberg</a>!</li>
<li>"Bare" returns from resources are now supported. You can skip the <code>return rep(data);</code> and just use <code>return data;</code> as long as you don't need any of the chaining methods (like <code>.withStatus()</code>). Pro-tip: you can use <a href="https://docs.taffy.io/#/3.3.0?id=querytoarray">queryToArray()</a> without <code>rep()</code>! Thanks to <a href="https://github.com/y2kiah">y2kiah</a> for the idea!</li>
<li>The dashboard and documentation for your Taffy API's got a nice sprucing up:
<ul>
<li>Larger text and increased contrast: better accessibility! 👍</li>
<li>Added a tabbed UI inside each resource's panel to separate the inline docs from the interactive runner. The docs always felt too squished over on the side before!</li>
<li>Added support for displaying sample return data for each method (more on this below).</li>
<li>With the addition of the sample responses, docs can get really long really fast, so I've added a small accordion to each method to collapse the documentation for its inputs and sample response (if any).</li>
</ul>
</li>
</ul>
<h2 id="sample-responses%2C-you-say%3F" tabindex="-1">Sample responses, you say?</h2>
<p>Yup! Have you ever tried to onboard a new API consumer and wanted to give them a way to interact with the API to verify they're calling it correctly, but either don't have real data for them yet or don't want them testing against it? Enter <strong>sample responses</strong> and <strong>simulated requests</strong>.</p>
<p>You specify the sample response for your GET method by adding a <code>sampleGetResponse</code> method that returns the raw data of your sample response. It doesn't receive any input arguments.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">sampleGetResponse</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token punctuation">[</span><br /> <span class="token punctuation">{</span><br /> <span class="token literal-property property">indentationMethod</span><span class="token operator">:</span> <span class="token string">'tabs'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">rating</span><span class="token operator">:</span> <span class="token string">'superior'</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">{</span><br /> <span class="token literal-property property">indentationMethod</span><span class="token operator">:</span> <span class="token string">'spaces'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">rating</span><span class="token operator">:</span> <span class="token string">'inferior'</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">]</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Your sample responses will automatically be displayed in a nice little collapsable drawer in the dashboard and docs pages.</p>
<p>If a simulated request is made, the sample response data is returned rather than running the request normally. If there is no applicable sample response method, the simulated request returns status 400.</p>
<p>By default simulated requests are made by adding <code>?sampleResponse=true</code> to your request (or e.g. <code>{"sampleResponse":true}</code> in a POST body...). You can override both the key and the password using the new <a href="https://docs.taffy.io/#/3.3.0?id=simulatekey">simulateKey</a> and <a href="https://docs.taffy.io/#/3.3.0?id=simulatepassword">simulatePassword</a> configuration attributes.</p>
<p>Of course there's <a href="https://docs.taffy.io/#/3.3.0?id=what39s-new-in-330">much more that's new and improved</a>, but those are the highlights. Enjoy!</p>
So Long ForgeBox2021-08-06T00:00:00Zhttps://adamtuttle.codes/blog/2021/so-long-forgebox/<p>Very quick note today. As of yesterday, <a href="https://github.com/atuttle/semaphore">Semaphore</a> has been removed from <a href="https://forgebox.io/">ForgeBox</a>. <a href="https://taffy.io/">Taffy</a> is not far behind.</p>
<p>I have no plans for them to return.</p>
<p>This is entirely for inter-personal reasons, which I don't want to get into here or now. The best summary of the situation is simply that cutting ties is better for my own mental health.</p>
<p>I apologize if this makes it more difficult for you to use Taffy or Semaphore. If you want to be notified of new versions, you might consider subscribing to release notifications on GitHub. Click the "Watch" button in the upper right, then click "Custom" in the list, and select the notifications you want to receive.</p>
<p><img src="https://adamtuttle.codes/img/2021/release-notifications.png" alt="How to get release notifications on GitHub" /></p>
<p>Or if that's not your thing, you could use <a href="https://followthatpage.com/">Follow That Page</a> to monitor the <a href="https://github.com/atuttle/Taffy/releases">Releases</a> page.</p>
Implementing Feature Flags with Semaphore2021-08-02T00:00:00Zhttps://adamtuttle.codes/blog/2021/implementing-feature-flags-with-semaphore/<p>Recently I wrote about <a href="https://adamtuttle.codes/tags/semaphore/">Semaphore</a>, a new open source cfml feature flags library I released. The library itself is simply a decision engine. You provide it a set of rules (native CFML data types arranged in a specific way), and then you can ask if any given rule is enabled for a user by providing (1) the rule name and (2) a data packet representing the user (again, native cfml data types).</p>
<p>At the time it was a prototype because it seemed like it should work fine but I hadn't actually implemented it in my application yet. Well, that's changed. We're running Semaphore in production now. 🥳</p>
<blockquote>
<p><strong>I'm not saying that it's bulletproof! We're a sample size of 1 with a trial run time measured in (single-digit!) days at the time of writing. But that's better than nothing.</strong></p>
</blockquote>
<p>As expected, using Semaphore required creating a <code>featureFlagService</code> in my application. That service is responsible for saving and loading the flag data (semaphore doesn't provide a storage mechanism), and it provides some helpers and wrappers to make working with feature flags very simple.</p>
<p>The reason it's not part of the Semaphore project is that it's specific to our framework (<a href="https://framework-one.github.io/">FW/1</a>), and our application architecture, so it won't be useful to you without significant modification. But I'm sure you want to see it anyway, so <a href="https://gist.github.com/atuttle/f695c8f27011ea2e7176e30f57d25a67">here's a gist of my entire featureFlagService.cfc</a>. If you want more explanation, keep reading!</p>
<p>In addition to this service, we also have some wiring to do in controllers. But I'm getting ahead of myself.</p>
<h2 id="how-to-create-your-own-featureflagservice" tabindex="-1">How to create your own featureFlagService</h2>
<p>What steps would you need to take if you were going to write your own <code>featureFlagService</code> specific to your application?</p>
<p>Let's start with the basics. On initialization, wait for dependencies to be wired, and then load the current flag data into memory. Here's my implementation:</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// featureFlagService.cfc</span><br /><br /><span class="token keyword">public</span> <span class="token keyword">function</span> <span class="token function">init</span><span class="token punctuation">(</span> <span class="token parameter">required utilService</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> variables<span class="token punctuation">.</span>utilService <span class="token operator">=</span> arguments<span class="token punctuation">.</span>utilService<span class="token punctuation">;</span><br /><br /> <span class="token comment">//load current feature flags</span><br /> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">refreshFlagsFromAPI</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token keyword">function</span> <span class="token function">refreshFlagsFromAPI</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> variables<span class="token punctuation">.</span>semaphore<span class="token punctuation">.</span><span class="token function">setAllFlags</span><span class="token punctuation">(</span> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">getAllFlagsFromAPI</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">private</span> struct <span class="token keyword">function</span> <span class="token function">getAllFlagsFromAPI</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">var</span> apiResponse <span class="token operator">=</span> variables<span class="token punctuation">.</span>utilService<span class="token punctuation">.</span><span class="token function">httpGet</span><span class="token punctuation">(</span> variables<span class="token punctuation">.</span>apiURL <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span> <span class="token operator">!</span><span class="token function">isJson</span><span class="token punctuation">(</span>apiResponse<span class="token punctuation">.</span>fileContent<span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">throw</span><span class="token punctuation">(</span>message<span class="token operator">:</span> <span class="token string">"Feature Flag API response is not valid json! 😱"</span><span class="token punctuation">,</span> <span class="token literal-property property">detail</span><span class="token operator">:</span> apiResponse<span class="token punctuation">.</span>fileContent<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">var</span> flags <span class="token operator">=</span> <span class="token function">deserializeJson</span><span class="token punctuation">(</span> apiResponse<span class="token punctuation">.</span>fileContent <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> flags<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>As you can see above, on initialization we're going to be making an HTTP request to get some data from an API. We're expecting a JSON response, and then when we get a valid response we call <code>semaphore.setAllFlags( flagData )</code>.</p>
<p>The code is organized this way so that we can run additional flag-data-updates on demand. <strong>That will be useful later.</strong></p>
<p>Now the application is loaded and the featureFlagService has flag data in memory. How do we evaluate a flag to see if it's on for the current user?</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// featureFlagService.cfc</span><br /><br /><span class="token keyword">public</span> boolean <span class="token keyword">function</span> <span class="token function">flagEnabled</span><span class="token punctuation">(</span> <span class="token parameter">required string flagId</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> variables<span class="token punctuation">.</span>semaphore<span class="token punctuation">.</span><span class="token function">checkForUser</span><span class="token punctuation">(</span> arguments<span class="token punctuation">.</span>flagId<span class="token punctuation">,</span> <span class="token function">getCurrentUserAttributes</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">getCurrentUserAttributes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> request<span class="token punctuation">.</span>featureFlagsUserAttributes<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Cool! So we can call <code>featureFlagService.flagEnabled( 'some_flag_name' )</code> from anywhere in our application and we'll get back a boolean that indicates whether we should treat the flag as on or off. But where does <code>request.featureFlagsUserAttributes</code> come from? 🤔</p>
<p>That's another one of those details that you're going to need to work out for yourself, BUT here's how we did it in our FW/1 application:</p>
<p>In our application, all controllers inherit from a <code>baseController.cfc</code> class, which we use to do things like enforce role-based authentication in a general way instead of re-implementing it for every action. This also happens to be a great place to stick something that runs early on in every request. In FW/1 applications, if your controller has a <code>before()</code> method, it gets called on every request <em>before</em> (hence the name) the controller method specific to the requested action. This is where we're setting <code>request.featureFlagsUserAttributes</code>.</p>
<p>So my <code>admin/baseController.cfc</code>'s <code>before()</code> method looks a little like this:</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// admin/baseController.cfc</span><br /><br /><span class="token keyword">public</span> <span class="token keyword">function</span> <span class="token function">before</span><span class="token punctuation">(</span> <span class="token parameter">rc</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//the requested action might not require user to be logged in</span><br /> <span class="token function">enforceLoginRequirements</span><span class="token punctuation">(</span> rc<span class="token punctuation">.</span>action <span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span> <span class="token function">userIsLoggedIn</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token function">loadUserAttributesForFeatureFlags</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">loadUserAttributesForFeatureFlags</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">var</span> cacheKeyName <span class="token operator">=</span> <span class="token string">'featureFlagsUserAttributes'</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span> <span class="token operator">!</span>request<span class="token punctuation">.</span><span class="token function">keyExists</span><span class="token punctuation">(</span> cacheKeyName <span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span> application<span class="token punctuation">.</span>sessionAdapter<span class="token punctuation">.</span><span class="token function">has</span><span class="token punctuation">(</span>cacheKeyName<span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> request<span class="token punctuation">[</span>cacheKeyName<span class="token punctuation">]</span> <span class="token operator">=</span> application<span class="token punctuation">.</span>sessionAdapter<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>cacheKeyName<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token keyword">else</span><span class="token punctuation">{</span><br /> <span class="token keyword">var</span> userAttributes <span class="token operator">=</span> featureFlagService<span class="token punctuation">.</span><span class="token function">buildUserAttributesFromAdminUser</span><span class="token punctuation">(</span> <span class="token function">getCurrentUser</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> application<span class="token punctuation">.</span>sessionAdapter<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span> cacheKeyName<span class="token punctuation">,</span> userAttributes <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> request<span class="token punctuation">[</span>cacheKeyName<span class="token punctuation">]</span> <span class="token operator">=</span> userAttributes<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">return</span> request<span class="token punctuation">[</span>cacheKeyName<span class="token punctuation">]</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Basically, if the user is logged in, then we're going to make sure the request variable is set. If it's not found in the request then we look for it in the user's session. Caching it in the session prevents re-calculation on every request; and the data is unlikely to change often enough to worry about it becoming stale. Worst case scenario the user has to log out and back in to see a change. 🤷♂️</p>
<p>The above snippet eventually calls <code>featureFlagService.buildUserAttributesFromAdminUser( getCurrentUser() )</code>, so what's in there? It's job is to take the given user and return the data structure that will be useful in rule evaluation:</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// featureFlagService.cfc</span><br /><br /><span class="token keyword">public</span> struct <span class="token keyword">function</span> <span class="token function">buildUserAttributesFromAdminUser</span><span class="token punctuation">(</span> <span class="token parameter">required user</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token punctuation">{</span><br /> <span class="token string-property property">'cust'</span><span class="token operator">:</span> configService<span class="token punctuation">.</span><span class="token function">getCustomer</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token string-property property">'env'</span><span class="token operator">:</span> configService<span class="token punctuation">.</span><span class="token function">getEnv</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token string-property property">'userId'</span><span class="token operator">:</span> user<span class="token punctuation">.</span><span class="token function">getUserId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token string-property property">'guid'</span><span class="token operator">:</span> user<span class="token punctuation">.</span><span class="token function">getGuid</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token string-property property">'firstName'</span><span class="token operator">:</span> user<span class="token punctuation">.</span><span class="token function">getFirstName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token string-property property">'lastName'</span><span class="token operator">:</span> user<span class="token punctuation">.</span><span class="token function">getLastName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span><br /> <span class="token string-property property">'roles'</span><span class="token operator">:</span> user<span class="token punctuation">.</span><span class="token function">getRoles</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>That almost covers everything you can see in my <code>featureFlagService.cfc</code>. In fact it does touch everything once. But there's one more mechanism that's important to the feature flags workflow that I haven't discussed yet: Updating flags and flag statuses in realtime without a deploy...</p>
<p>You know... the <em><strong>entire point</strong></em> of feature flags.</p>
<p>It's really quite simple actually. In the same <code>admin/baseController.cfc</code>, still in the <code>before()</code> method, we have a listener that checks for a specific URL param, and if found, does an immediate on-demand refresh:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token comment">// admin/baseController.cfc</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">public</span> <span class="token keyword">function</span> <span class="token function">before</span><span class="token punctuation">(</span> <span class="token parameter">rc</span> <span class="token punctuation">)</span><span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token comment">//the requested action might not require user to be logged in</span></span><br /><span class="highlight-line"> <span class="token function">enforceLoginRequirements</span><span class="token punctuation">(</span> rc<span class="token punctuation">.</span>action <span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><mark class="highlight-line highlight-line-active"> <span class="token comment">// I've changed the parameter name/values in case anyone gets any wise ideas... 🤨</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">if</span> <span class="token punctuation">(</span> url<span class="token punctuation">.</span><span class="token function">keyExists</span><span class="token punctuation">(</span><span class="token string">'peanutButter'</span><span class="token punctuation">)</span> <span class="token operator">&&</span> url<span class="token punctuation">.</span>peanutButter <span class="token operator">==</span> <span class="token string">"jellyTime"</span> <span class="token punctuation">)</span><span class="token punctuation">{</span></mark><br /><mark class="highlight-line highlight-line-active"> featureFlagService<span class="token punctuation">.</span><span class="token function">refreshFlagsFromAPI</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">}</span></mark><br /><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span> <span class="token function">userIsLoggedIn</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token function">loadUserAttributesForFeatureFlags</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>In our flag-data admin interface, after flags are updated, we send http requests to each server to notify them that flags have been updated and they should pull down the updated data. None of that is pictured here, because this post doesn't include any portion of my flag-data-admin interface; but hopefully you can see how that should work.</p>
<h2 id="what's-still-missing%3F" tabindex="-1">What's still missing?</h2>
<p>I have very much glossed over a few things:</p>
<ul>
<li>flag data storage</li>
<li>flag data api</li>
<li>flag data admin interface</li>
</ul>
<p>That's because these things are extremely specific to your application and your environment. I've talked a little bit on <a href="https://workingcode.dev/episodes/032-what-comes-after-senior-developer/">Working Code Podcast episode 32</a> about the storage mechanism we're using, but I won't be going into it here.</p>
<p>The API is a simple JSON responder with the current value of the flags data file. It's built into the flag administration interface.</p>
<p>And to be honest, the administration interface I built for managing our flags is <em>extremely</em> crude. I actually got a compliment from a coworker praising me for doing only the bare minimum to get it working -- it's very obviously a developer tool, not something polished enough for non-tech people to mess with. Of course I accepted the compliment as if that was my plan all along. 😳</p>
REST Assured Paperbacks Now Available!2021-07-27T00:00:00Zhttps://adamtuttle.codes/blog/2021/rest-assured-paperbacks-now-available/<p>The wait is finally over! I published the first edition of REST Assured (then called, imaginatively, <a href="https://adamtuttle.codes/blog/2014/rest-web-apis-the-book/">"REST Web API's: The Book"</a>) in December of 2014, and immediately there were people who wanted a physical copy. <em>And some of them weren't my mom!</em></p>
<p>It's been a long time and several revisions of the book, but for the first time a paperback is now available to the public:</p>
<p><img src="https://adamtuttle.codes/img/2021/rest-assured-paperbacks.jpg" alt="Adam Tuttle holding several paperback copies of his book, REST Assured" /></p>
<p style="text-align: center"><a class="btn" href="https://www.lulu.com/en/us/shop/adam-tuttle/rest-assured/paperback/product-en5e7n.html">Buy a REST Assured Paperback now</a></p>
<p>Just in case there were any questions: All proceeds from sales of my book are used to fund my frivolous hobbies of skydiving and woodworking. I wouldn't want you worrying that I might do something useful with it.</p>
<p>As ever, everyone who buys a copy has <strong>my unending gratitude.</strong> Thank you so much! 🥰 Self-publishing has been an enlightening and sometimes frustrating journey, but I wouldn't do it any other way.</p>
<p>Lastly, I'll mention that I'm happy to make signed copies available, if that interests you. It takes longer to get to you, and I have to add a few bucks for the 2nd shipping journey, but if that hasn't scared you off, you can send me a <a href="https://twitter.com/adamtuttle">DM on Twitter</a> or email me at this-blogs-domain at gmail.</p>
Next.js Docker Single Entrypoint for Dev and Prod2021-07-06T00:00:00Zhttps://adamtuttle.codes/blog/2021/nextjs-docker-single-entrypoint-for-dev-and-prod/<p>Maybe I shouldn't be as excited about this as I am, but I love it when simple things do smart things.</p>
<p>I'm currently working on a Next.js application, which will be deployed in a Docker container on AWS Fargate. When you generate a new Next.js application, you get these scripts in your <code>package.json</code> to start:</p>
<pre class="language-json"><code class="language-json"><span class="token property">"scripts"</span><span class="token operator">:</span> <span class="token punctuation">{</span><br /> <span class="token property">"dev"</span><span class="token operator">:</span> <span class="token string">"next dev"</span><span class="token punctuation">,</span><br /> <span class="token property">"build"</span><span class="token operator">:</span> <span class="token string">"next build"</span><span class="token punctuation">,</span><br /> <span class="token property">"start"</span><span class="token operator">:</span> <span class="token string">"next start"</span><span class="token punctuation">,</span><br /> <span class="token property">"lint"</span><span class="token operator">:</span> <span class="token string">"next lint"</span><br /><span class="token punctuation">}</span><span class="token punctuation">,</span></code></pre>
<p>As you might expect, you use <code>npm run dev</code> to start a local development instance, which is not optimized for production (it assumes you're making lots of changes and is more eager to recompile stuff, basically). In production, you <em>should</em> be starting the application with <code>npm run start</code>. This assumes that you've already created a production-optimized build with <code>npm run build</code>, which can make assumptions like you're not going to be changing things, so it should pre-compile as much as possible.</p>
<p>So, your typical Dockerfile for a Next.js application should resemble this:</p>
<pre class="language-dockerfile"><code class="language-dockerfile"><span class="token instruction"><span class="token keyword">FROM</span> node:LTS <span class="token keyword">as</span> builder</span><br /><span class="token instruction"><span class="token keyword">WORKDIR</span> /usr/src/app</span><br /><span class="token instruction"><span class="token keyword">COPY</span> . /usr/src/app</span><br /><span class="token instruction"><span class="token keyword">ENV</span> NODE_ENV=production</span><br /><span class="token instruction"><span class="token keyword">RUN</span> npm ci</span><br /><br /><span class="token instruction"><span class="token keyword">FROM</span> node:LTS-alpine</span><br /><span class="token instruction"><span class="token keyword">WORKDIR</span> /usr/src/app</span><br /><span class="token instruction"><span class="token keyword">COPY</span> <span class="token options"><span class="token property">--from</span><span class="token punctuation">=</span><span class="token string">builder</span></span> /usr/src/app /usr/src/app</span><br /><span class="token instruction"><span class="token keyword">RUN</span> npm run build</span><br /><span class="token instruction"><span class="token keyword">CMD</span> npm run start</span></code></pre>
<p>This is all pretty standard stuff. We're using a multi-stage build to minimize final container size, and creating a production build. (For what it's worth, I initially had the <code>npm run build</code> step in the first stage, but the build artifacts weren't found when running <code>npm run start</code> in the final container, even with the full app directory copy. 🤷♂️)</p>
<p>This is where it gets interesting.</p>
<p>I've written in the past about <a href="https://adamtuttle.codes/blog/2021/my-ongoing-love-affair-with-gnu-make/">my love of Makefiles</a>. So now let's assume I have a Makefile with the following target, which I use to start my container for local development.</p>
<pre class="language-makefile"><code class="language-makefile"><span class="token target symbol">dev-up</span><span class="token punctuation">:</span> .built<br /> docker run --rm -d --name widgets-dev \<br /><span class="token target symbol"> -p 3000</span><span class="token punctuation">:</span>3000 \<br /><span class="token target symbol"> -v `pwd`</span><span class="token punctuation">:</span>/usr/src/app \<br /> --env NODE_ENV<span class="token operator">=</span>development \<br /><span class="token target symbol"> myco/widgets</span><span class="token punctuation">:</span>dev</code></pre>
<p>This starts the container with the name <code>widgets-dev</code>, mapping host port 3000 to container port 3000, NODE_ENV set to development, <strong>and crucially, a volume of the current working directory mounted as <code>/usr/src/app</code></strong>.</p>
<p>Initially I expected this to <em>just work</em>. Since the working directory is mounted inside the container at the application source directory, fast refresh and all of the usual development-environment goodies should still work out of the box, right?</p>
<p>Well, can you see what's missing? Yes, we're properly overriding the <code>NODE_ENV</code> value when we start the container for development, but the container has only one <code>CMD</code> entrypoint, and we want it to do different things depending on its current environment variables. In dev it should run <code>npm run dev</code>, but in production or qa it should run <code>npm run start</code>.</p>
<p>I found an easy and simple way to accomplish this with npm scripts, but it's not perfect...</p>
<h2 id="my-solution" tabindex="-1">My Solution</h2>
<blockquote>
<p>⚠️ NOTE: Since writing this section I've since found a method that's <em>so</em> much better! You can keep reading to learn something that could be useful to know elsewhere, but I would now recommend that you <a href="https://adamtuttle.codes/blog/2021/nextjs-docker-single-entrypoint-for-dev-and-prod/#update-an-even-better-way">use this technique in your npm scripts</a>.</p>
</blockquote>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token property">"scripts"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token property">"dev"</span><span class="token operator">:</span> <span class="token string">"next dev"</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token property">"build"</span><span class="token operator">:</span> <span class="token string">"next build"</span><span class="token punctuation">,</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token property">"prestart"</span><span class="token operator">:</span> <span class="token string">"node -e 'process.env.NODE_ENV === \"production\" || process.exit(1);' || npm run dev"</span><span class="token punctuation">,</span></mark><br /><span class="highlight-line"> <span class="token property">"start"</span><span class="token operator">:</span> <span class="token string">"next start"</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token property">"lint"</span><span class="token operator">:</span> <span class="token string">"next lint"</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">,</span></span></code></pre>
<p>Here I've added a script named <code>prestart</code> which <a href="https://docs.npmjs.com/cli/v6/using-npm/scripts#pre--post-scripts">npm will execute before it runs the <code>start</code> script</a>. My <code>prestart</code> script uses the inline-evaluation feature of node.js (<code>-e</code>) to check the current value of <code>NODE_ENV</code> and if it's anything other than <code>production</code> the inline script exits with error code 1. After the inline script I have <code>|| npm run dev</code> which will execute the <code>npm run dev</code> only if the previous command exited with an error code.</p>
<p>This is working reasonably well for me right now. The container always runs <code>npm run start</code> as its entrypoint, and it runs in dev mode in the dev environment, or production mode in the production environment.</p>
<p>Here's where it's not perfect:</p>
<p>In local development, when the dev server shuts down, npm has been waiting to spring into action and it follows suit by trying to execute <code>npm run start</code>. Remember that the dev server has been running as part of <code>npm run prestart</code> this entire time. I'm not sure yet if there's a way to stop this from happening, and to be fair it's <em>not a big deal</em> because the error happens inside the docker container which hides it from view.</p>
<p>I know that I can make the <code>npm run dev</code> portion of <code>prestart</code> exit with a nonzero code and that this would stop npm from running the <code>start</code> script, but it does then return an error:</p>
<pre class="language-json"><code class="language-json"><span class="token property">"prestart"</span><span class="token operator">:</span> <span class="token string">"node -e 'process.env.NODE_ENV === \"production\" || process.exit(1);' || (npm run dev && exit 1)"</span></code></pre>
<p>I'm not sure if that's trading one bad thing for another, nor whether or not it's a good trade to make.<span id="update-an-even-better-way"></span></p>
<h2 id="%E2%9D%97update%3A-an-even-better-way" tabindex="-1">❗UPDATE: An Even Better Way</h2>
<p>Here's an even better way which has all of the same benefits, with no draw-backs, and simpler and <em>less</em> code.</p>
<pre class="language-json"><code class="language-json"><span class="highlight-line"><span class="token property">"scripts"</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token property">"build"</span><span class="token operator">:</span> <span class="token string">"next build"</span><span class="token punctuation">,</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token property">"start"</span><span class="token operator">:</span> <span class="token string">"npm run start:$NODE_ENV"</span><span class="token punctuation">,</span></mark><br /><span class="highlight-line"> <span class="token property">"start:"</span><span class="token operator">:</span> <span class="token string">"npm run start:development"</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token property">"start:development"</span><span class="token operator">:</span> <span class="token string">"next dev"</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token property">"start:production"</span><span class="token operator">:</span> <span class="token string">"next start"</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token property">"start:test"</span><span class="token operator">:</span> <span class="token string">"next start"</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">,</span></span></code></pre>
<p>Npm will evaluate any environment variables you include in the script, so it's really easy to make sub-scripts that are environment-specific. Here, I've created <code>start</code> which you trigger with <code>npm run start</code> as usual. It will run <code>npm run start:$NODE_ENV</code>, so it will ultimately run <code>start:production</code> if your <code>NODE_ENV</code> is <code>production</code>. On the off chance that you don't have a <code>NODE_ENV</code> value set, I've also created a <code>start:</code> script which then proxies on to <code>start:development</code>.</p>
<p>This method doesn't rely on any "clever" tactics to interrupt the original intent, doesn't result in any swallowed errors, and just runs smoothly no matter what. And it's <em>dead simple</em>.</p>
On Resisting the Urge to Rush Testing2021-06-30T00:00:00Zhttps://adamtuttle.codes/blog/2021/on-resisting-the-urge-to-rush-testing/<p>I am very much in favor of automated testing. I understand the benefits and I need them in my life. But if I'm being honest, I know that the confidence of a passing test suite is only as good as the <em>effort</em> that went into writing it. So then it stands to reason that one of the most important tenets of the testing ethos should be something like, "Slower is better if you identify more test cases."</p>
<h2 id="passing-tests-prove-only-that-the-remaining-bugs-haven't-been-found" tabindex="-1">Passing tests prove only that the remaining bugs haven't been found</h2>
<blockquote>
<p>"Program testing can be used to show the presence of bugs, but never to show their absence!" — Edsger W. Dijkstra</p>
</blockquote>
<p>Do you recall that application that I was writing and <a href="https://adamtuttle.codes/tags/testing/">trying to use to experiment with TDD</a>? Here's something I didn't tell you at the time.</p>
<p>I was feeling very fatigued from so much additional test writing, and the learning necessary to write my tests well. I had pre-written a couple of dozen test stories and I was laser-focused on writing tests for them, and on reaching 100% code coverage, and as a result I occasionally ignored when I would find a bug in the code that wasn't covered by my tests.</p>
<p>To rephrase:</p>
<p>Occasionally, as I was trying to get my tests to prove the thing they were intended to prove, I would accidentally trip over a bug in the code. And instead of writing an extra test to prove that there was a bug ("red"), I jumped ahead to fixing the bug ("green") so that I could get back to writing the test at hand.</p>
<p>I let my fatigue and my desire to be done take precedence over doing the correct thing.</p>
<p>It's too late now. I can't remember what the bugs were, and they're fixed in the code. It would be possible to go back and write additional tests to prove that those bugs aren't in the code, if I could remember what they were, but I don't. And that's where I failed myself.</p>
<p>This time.</p>
<p>The lesson I choose to take away from this experience is that, <em>yes, testing well is <strong>hard</strong> and <strong>slow</strong>, at first.</em> BUT! It's important.</p>
<p>Nobody denies that you've got to crawl before you can walk, and you've got to walk before you can run. That's common sense. But when you can already write good, clean code quickly and you intentionally slow yourself down by learning to write tests at the same time... the pain is real! It takes conscious and sometimes vigorous effort to do the right thing, slow down, and write the tests you weren't planning to write.</p>
<p>I had a friend who is very much farther ahead of me on the testing trail read the first draft of this article and he not only agreed with what I've said, but also admitted that sometimes he does this too. So we're not alone. To err is human.</p>
<p>We're learning to walk, here. We're going to have some exciting moments of success, and we're also going to fall flat on our butts sometimes. Do the best you can in the moment, and take time later to reflect on what you could have done better. (This article is me reflecting!)</p>
<p>The next time I'm feeling test-fatigued, I should give myself a break. Go for a walk, pet the dog, have a conversation with my kids... Something to put more fuel in the tank and give me the willpower to continue fighting the good fight.</p>
<p>If we want our companies to be willing to give us the extra time to write tests, we have to be willing to write them all — at least the ones we know about.</p>
Testing Media Roundup #12021-06-11T00:00:00Zhttps://adamtuttle.codes/blog/2021/testing-media-roundup-1/<p>The project I was using to push myself forward on testing stuff? I wrapped that up at the end of last week. It's by no means perfect, but it's better than it would have been without tests, and it works, and it's on its way to production. I would love to get the opportunity to go back and make the tests themselves better, cleaner, and smarter. I've already learned a few things that I would do differently next time.</p>
<p>But in the meantime, my ongoing projects aren't tested with automated-tests (for <em>*reasons*</em>) and life must go on. Momentum takes lots of different forms. So instead of pondering testing while working, this week I tried to consume a bunch of testing content from others. My podcast cohosts suggested that I should share that content here, for people who might be interested in it but who wouldn't have otherwise seen it. I'm happy to oblige. Maybe this will become a regular thing? 🤷♂️</p>
<h2 id="are-we-all-doing-tdd-wrong%3F" tabindex="-1">Are we all doing TDD wrong?</h2>
<p>Firstly, here's a TDD conference presentation video I enjoyed. He ran out of time towards the end, just when I thought he was getting to the good stuff, so I think this could have benefited from another 15 minutes, but it definitely succeeded at churning something up inside me. I will definitely need to watch this again and try to figure out how I should be changing my workflows.</p>
<p>In particular I was struck by the way he stresses the red-green-refactor ("RGR") cycle and the specific benefits available from doing it right. I think until now I was treating RGR as a quick checkbox to be able to say I was doing TDD and not really a process that derives benefit.</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/EZ05e7EMOLM" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
<h2 id="what-is-an-%22integration%22-test%2C-anyway%3F" tabindex="-1">What is an "integration" test, anyway?</h2>
<p>And perhaps more interestingly, is it different depending on the types of systems you're testing?</p>
<p>I'm a big fan of <a href="https://kentcdodds.com/">Kent C. Dodds</a> and his <a href="https://kentcdodds.com/blog/">blog</a>, his <a href="https://kentcdodds.com/courses">courses</a>, and his <a href="https://github.com/kentcdodds/">open source work</a>. Some of the stuff he's been teaching came under scrutiny recently by <a href="https://www.tbray.org/ongoing/When/202x/2021/05/15/Testing-in-2021">Tim Bray</a>, and then by <a href="https://martinfowler.com/articles/2021-test-shapes.html">Martin Fowler</a>. Kent responded with an explanation of <a href="https://kentcdodds.com/blog/the-testing-trophy-and-testing-classifications">the way he thinks about this problem</a>, and I think he does a great job turning the conversation toward something useful.</p>
<p>At the end of the day, does it matter if we disagree on what makes something an "integration test"? Does it matter if I have too many "integration" tests in your opinion? I think not. Anything that encourages people to write more tests, and helps them figure out how to do that well, is just fine with me.</p>
<h2 id="i-got-the-book" tabindex="-1">I got the book</h2>
<p><a href="https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530?crid=1D8X147Q3MS5G&dchild=1&keywords=tdd+by+example&qid=1623418032&sprefix=tdd+by+example%2Caps%2C136&sr=8-3&linkCode=li3&tag=tuttl-20&linkId=f4d8afaabf721a86be63f5b8d30c708a&language=en_US&ref_=as_li_ss_il" target="_blank"><img border="0" src="https://adamtuttle.codes/img/2021/tdd-by-example-cover.jpg" /></a><img src="https://ir-na.amazon-adsystem.com/e/ir?t=tuttl-20&language=en_US&l=li3&o=1&a=0321146530" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /></p>
<p>From what I understand, if TDD has a Bible it is <a href="https://amzn.to/35hV6X3">Test-Driven Development By Example, by Kent Beck</a>. So I bought myself a copy. I haven't even cracked the cover open yet, but I'm trying to take this aspect of my professional development seriously so I'm quite looking forward to better understanding TDD. I don't know about you, but I have a really bad habit of hearing a few people talk about a concept —like TDD— and then fooling myself into believing that I understand it enough to do it, without any sort of training formal or otherwise.</p>
<p>It's time to admit to myself that I don't <em>actually</em> know much about TDD, and that I could benefit from learning it from the TDD Bible. And then, you know... Do it.</p>
The Flywheel of Testing2021-06-04T00:00:00Zhttps://adamtuttle.codes/blog/2021/the-flywheel-of-testing/<p>On my continuing quest to get better at testing, I have spent quite a lot of time in the last couple of weeks reading about testing, watching tutorial videos, and practicing testing in my work. It has been a long, slow, difficult, slog. I'm not sure what made me think of it, but I was reminded of a flywheel.</p>
<p><img src="https://adamtuttle.codes/img/2021/flywheel.jpg" alt="Flywheel" /></p>
<p>If you're not familiar, a flywheel uses its inertia (the conservation of angular momentum) to store energy. There are flywheels in <acronym title="Internal Combustion Engine">ICE</acronym> cars, childrens toys, and thousands of industrial machines. They are all around us. The dimensions and mass of the flywheel determine the energy needed to get it spinning and up to a useful speed, but once it's there it helps even out any fluctuation in pace.</p>
<h2 id="so-too-(i-am-learning)-with-testing" tabindex="-1">So too (I am learning) with testing</h2>
<p>I believe that if not already fully, I am <em>*this close*</em> to a complete understanding of the fundamentals of testing. As concepts, I grok pure functions, side effects, isolation, dependency injection, and mocking.</p>
<p>What I don't fully grok yet is <em>the tools I'm using</em> to put this understanding to useful implementation. I am not exactly proud of the test code I've been writing this week. It's a mess.</p>
<p>But it works.</p>
<p><img src="https://adamtuttle.codes/img/2021/passing-tests.png" alt="My passing test suite" /></p>
<p>That's 100% passing tests with more than 96% code coverage overall. And I believe I'm going to hit 100% very soon.</p>
<p>I'm proud that it exists. I'm proud that it does its job. I would never dream of sharing the test code as an example of how anyone should write tests. Well, maybe just to show that it's OK to do a bad job on your first draft. Just like writing research papers in school, and just like writing our application logic, it doesn't have to be perfect in your first draft.</p>
<p>The practice of "write & refactor & write & refactor & write & refactor & ..." is well ingrained in me when it comes to prose. I use it with my blog articles, emails, even the terrible fiction I occasionally write that will <em>never</em> see the light of day. I find it really helpful to be able to get an idea out of my head and in front of my eyes without worrying so much about grammar and clarity. It's a lot easier to refactor an existing sentence to use proper grammar and to be more clear than it is to divine a perfect sentence without ever putting (metaphorical) pen to paper.</p>
<p>Some of the code I write today comes out pretty decently formed on the first try, but only for languages that I've been using ~daily for ~decades, and even then, only in certain contexts.</p>
<p>If testing has a flywheel, and I say that it does, the energy it consumes to get up to speed is donated by your doing a bad job writing tests, but writing them anyway. Write them, make them useful, then purposefully inspect them and think about how to make them better. It's going to take longer than it should, at first. Like, way longer. That's ok. It's worth it. Just as our code needs some cycles of write & refactor, so too do our tests.</p>
<p>Mine do, at least.</p>
<p>Just as perpetual motion machines don't exist, getting your testing flywheel up to speed doesn't mean it will generate free tests into perpetuity with zero effort. The job of the flywheel is to slow the draining of energy from the system, and to smooth out any unevenness in inputs. If your testing flywheel is fully up to speed and you take 2 weeks of vacation from work, there's a good chance you're going to come back and be able to get right back to work without much effort to get back into the rhythm.</p>
<p>The inverse is also true: If your flywheel is only barely moving before you take that same 2 weeks of vacation, coming back there's a much higher likelihood that your flywheel has slowed to a complete stop and you're going to have to start over.</p>
<p>If you have the time, open source projects (even if they aren't likely to be put to use) are a great low-risk opportunity to learn new techniques and tools. Write a blackjack game, and use it to learn to write tests. It won't matter if you get stuck because nobody is depending on your blackjack game, and when you do figure it out you'll have new skills you can put to use at your day job.</p>
<p>Don't expect testing to be something you decide to start doing and instantly find yourself able to do up to even your own meager standards. Getting that flywheel moving takes time. Lots of it.</p>
<p><em>And you know unclean code when you see it</em>. That's what makes it hurt on the inside, and I bet it's what turns a lot of would-be testers away. They couldn't push through the pain of having to write bad tests so that they could learn to write good tests.</p>
<p>Plan for that extra time and effort when you start to implement testing in your work. But also expect that it gets easier if you keep going. Eventually you should start to see its benefits and the inertia stored in your testing flywheel will help you keep that testing momentum smooth and even.</p>
<p>Write & refactor & write & refactor & write & refactor & ...</p>
My Jest Mock Calls Are Missing Data. Now What?2021-06-03T00:00:00Zhttps://adamtuttle.codes/blog/2021/my-jest-mock-calls-are-missing-data-now-what/<p>Yesterday I had an altogether too familiar experience. I spent several hours debugging, and once I believed that the answer was beyond me, I started drafting an explanation and code samples that would eventually become a Stack Overflow question. After 20 minutes of explaining the problem to my hypothetical Stack Overflow helpers, the answer came to me.</p>
<p><a href="https://twitter.com/AdamTuttle/status/1400201226625691652">https://twitter.com/AdamTuttle/status/1400201226625691652</a></p>
<p>I've already spoiled the solution above, but what was the problem, and how can I share that knowledge in a way to help others who might run into the same situation?</p>
<ul>
<li>I'm using <a href="https://jestjs.io/">Jest</a> for testing some JavaScript code</li>
<li>The code uses fetch to make HTTP requests, and I'm mocking fetch to make those requests controllable and fast</li>
<li>It appeared that fetch requests <em>that I knew were happening!</em> weren't showing up in the <code>fetch.mock.calls</code> array.</li>
</ul>
<p>Wait, what? How can I know that the calls were made and also not see them in the <code>fetch.mock.calls</code> array?</p>
<p>Well, my mock looks like this:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">fetch<span class="token punctuation">.</span><span class="token function">mockResponse</span><span class="token punctuation">(</span><span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token parameter">req</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>req<span class="token punctuation">.</span>url<span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>req<span class="token punctuation">.</span>url <span class="token operator">===</span> <span class="token string">'...'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">return</span> <span class="token constant">JSON</span><span class="token punctuation">.</span><span class="token function">stringify</span><span class="token punctuation">(</span> <span class="token comment">/* ... */</span> <span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>As my test ran, I could see about half a dozen HTTP requests logged by the <code>console.log</code> statement. Ok, great, but...</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> thingDoer <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'./thingDoer'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'does the thing'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token function">thingDoer</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><mark class="highlight-line highlight-line-active"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>fetch<span class="token punctuation">.</span>mock<span class="token punctuation">.</span>calls<span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token function">expect</span><span class="token punctuation">(</span>fetch<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toHaveBeenCalledWith</span><span class="token punctuation">(</span> <span class="token comment">/* ... */</span> <span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>How is it, then, that the <code>console.log</code> in the test shows only 1 call in the array?!</p>
<p>Well, maybe you've already guessed it from the spoiler in the tweet, but the answer was as simple as a missing <code>await</code>. Specifically, the <code>thingDoer</code> method is async and returns a promise.</p>
<p>Since I wasn't awaiting that promise, <code>thingDoer</code> returned early and I was running my <code>console.log</code> and expectations <em>while it was still running</em>. That explains why the log of <code>fetch.mock.calls</code> had less data than I could see in the individual logs for each request... they hadn't been made <strong>yet</strong>.</p>
<p>Making my test async and awaiting <code>thingDoer</code> solved it.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> thingDoer <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'./thingDoer'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'does the thing'</span><span class="token punctuation">,</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">await</span> <span class="token function">thingDoer</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token function">expect</span><span class="token punctuation">(</span>fetch<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toHaveBeenCalledWith</span><span class="token punctuation">(</span> <span class="token comment">/* ... */</span> <span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Hopefully this helps someone else out there going nuts because things that they can see happening aren't in the list of things that happened. But let's be realistic. It'll be me referring to this article again in 9 months. 🤷♂️</p>
TDD: Getting Out of My Own Way2021-05-27T00:00:00Zhttps://adamtuttle.codes/blog/2021/tdd-getting-out-of-my-own-way/<p>Yesterday I wrote about how I was <a href="https://adamtuttle.codes/blog/2021/lip-writing-testable-nodejs-code/">trying to do TDD for a JavaScript project but I got stuck</a>. I'm not stuck any more, thanks to a comment left by <a href="https://m5ls5e.com/">Michael Sprague</a>.</p>
<p>Here's the thing. <strong>I wasn't stuck on something technical.</strong> And that makes it all the more embarassing that I had spent half of my workday trying to get past it.</p>
<p>I was stuck on something philosophical. I didn't want to do the obvious thing because it seemed messy, which would be <acronym title="at a right angle to; at odds with">orthogonal</acronym> to <a href="https://workingcode.dev/episodes/022-book-club-1-clean-code-by-uncle-bob-martin-pt1/">"Clean Code"</a>. I recently read Uncle Bob's infamous tome and it felt wrong to intentionally write code that I knew I didn't like.</p>
<p><img src="https://adamtuttle.codes/img/2021/stuck.jpg" alt="A van stuck in the mud" /></p>
<p>I was forgetting that, "Good code isn't written, it's refactored." This is something that my podcast cohost Ben recently quoted from <a href="https://www.bikeshed.fm/294">the last episode of The Bike Shed podcast</a>. I should get this phrase tattoed backwards on my forehead so that I have to look at it in the mirror while I'm brushing my teeth every day.</p>
<p>To recap, I had a module with a helper function that I wanted to be able to test in isolation. I wanted my module to export only the main function, and I found the idea of creating a separate module for the helper function unappealing because I thought that establishing this pattern would result in the number of helper function modules spiraling beyond all bounds of reasonability in no time flat.</p>
<p><strong>Node.js has a solution for this problem.</strong> Modules can export a map (or strucure/object, if you prefer) and those maps can have as many properties and methods as you like. In tutorials you so often see these simply called <code>utils.js</code> or <code>helpers.js</code>.</p>
<p>I have grown to <em>hate</em> that approach to module naming. It feels lazy to me. Like naming a variable <code>myVar</code>. My lizard brain criticizes, <em>If you weren't so lazy, you could actually come up with a useful and helpful name for that variable/module.</em> And worse yet, given 10 seconds, I couldn't come up with a useful name for a collection of helpers for my module.</p>
<p>I had two obvious solutions in front of me, and I was subconsciously avoiding both of them.</p>
<p>I could either create an extra module for my helper methods to live in, or I could give up my arbitrary <em>"I only want this module to export its public api"</em> requirement. Either solution would make the helper methods testable and available to my module without proliferating a billion tiny modules.</p>
<p>Why was I fighting so hard against the idea of a billion tiny modules? Isn't that one of the supposed strengths of Node.js? Well, yes and no. Imagine a system with (more realistically) 2,000 helper function modules. Now you need to find the one that you think has a bug in it. Where is it? It's probably in a messy folder with 1,999 of its friends, right?</p>
<p>In my opinion, the "billions of tiny modules" approach is still possible to accomplish cleanly, but it would necessitate extreme care in naming and organization in order to be clean. Naming things is one of the hardest problems in programming, right? That's a lot of cognitive load that you could spend elsewhere if you didn't have to do it. And with modern IDE's finding things isn't that hard. But what remains hard is that for every new helper-function-module you create you then have to take the time to name it well and put it in a logical place within your project, a couple of time penalties that I'd rather avoid if I can.</p>
<p>But more importantly, I was resisting the urge to create individual helper modules or helper function group modules because I was trying to hold myself to a minimum standard of "purity" during the writing process. But as I so deftly demonstrated — I got stuck on literally the first test I tried to write! — that is not a productive line of thinking. I'm probably very poorly cribbing this from <strong>Clean Code</strong> or some smart person on Twitter or something, but I think of the steps of writing good code as:</p>
<ol>
<li>write the code</li>
<li>make it work</li>
<li>make it correct</li>
<li>make it fast (enough)</li>
<li>make it clean</li>
</ol>
<p>There's no sense focusing on code cleanliness during step 1, because it's going to be refactored several times anyway, and cleanliness can be refactored in later.</p>
<p>I expect that I will get better at the "brainstorming" phase of coding as part of the TDD process with more practice. At the moment, the two goals sometimes feel antithetical to one another.</p>
<hr />
<p>I ended yesterday's article saying that I had no choice but to set aside my TDD goals and press forward on the project — at least until I had the testing stuff figured out (running in a parallel mental thread).</p>
<p>So I did that. And I made quite a lot of progress on my project, and now the code is pretty far ahead of the tests. And some choices I made along the way are definitely going to need to be refactored to make them more testable, and cleaner. And that's ok.</p>
LIP: Writing Testable Node.js Code2021-05-27T00:00:00Zhttps://adamtuttle.codes/blog/2021/lip-writing-testable-nodejs-code/<p>Yesterday I was trying to <acronym title="Test Driven Development">TDD</acronym> some Node.js code, and I got stuck. I asked for help on the <a href="https://kentcdodds.com/discord">KCD Discord</a><sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2021/lip-writing-testable-nodejs-code/#fn1" id="fnref1">[1]</a></sup> but I guess my explanation of the problem wasn't great, so I promised I would follow up with a repro case. Meanwhile I am also trying to do better at <a href="https://www.swyx.io/learn-in-public/">learning in public</a>, so I'm going to write about it here, too. So here we go.</p>
<p>I am working on an AWS Lambda service that will run on a schedule and process some data. It will be written in Node.js. Node Lambda functions define a starting point file that exports an object containing a function. I think the convention is <code>index.js</code> and <code>{ handler: () => {} }</code>, but it's configurable. I'll stick with the convention. Also, Lambdas can either report that they are complete via a callback or a promise. I'm a big fan of async/await, so I'm going to go with that.</p>
<p>In order to make my module more testable I thought I would start by creating a module that the handler calls, so the index.js is quite simple:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> redisClickLogger <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'./lib/redis-click-logger'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br />module<span class="token punctuation">.</span>exports <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token function-variable function">handler</span><span class="token operator">:</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token keyword">await</span> <span class="token function">redisClickLogger</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p>Fine so far. Now let's look at some pseudocode of what the handler has to do. Its job will be to (1) grab a batch of click data from Redis and write it to the database, and then (2) send alerts based on that click data and some configuration of when to send each alert.</p>
<p>As I said, I was trying to do TDD. TDD purists, please forgive me. I know that I should have already written a test or two before even getting this far, but I'm new to this and I need to figure out the mechanics of my testing tools (pretty much the entire premise for this article) before I can attempt to do it "right." I've gotta walk before I can run.</p>
<p>Anyway, at this point I thought through my requirements and wrote a bunch of stories in the form of placeholder test cases:</p>
<pre class="language-js"><code class="language-js"><span class="token function">describe</span><span class="token punctuation">(</span><span class="token string">'redis-click-logger'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'loads batch size from ENV vars'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'loads target environment id from ENV vars'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'loads config via http api'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'skips the localdev customer'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'does not run for customers with email disabled'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'batches notification setting reads'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'batches click activity writes'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'only processes alerts once per message per batch'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>Seems like a good enough place to start. In theory the first two tests were easy enough to write; you can just set a value into <code>process.env</code>, run the function that loads it, and then check the actual value against the expected value:</p>
<pre class="language-js"><code class="language-js"><span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'loads batch size from ENV vars'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> testVal <span class="token operator">=</span> <span class="token punctuation">(</span>Math<span class="token punctuation">.</span><span class="token function">random</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">*</span> <span class="token number">1000</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toFixed</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> process<span class="token punctuation">.</span>env<span class="token punctuation">.</span><span class="token constant">BATCH_SIZE</span> <span class="token operator">=</span> testVal<span class="token punctuation">;</span><br /> <span class="token comment">// todo: where are we getting the getBatchSize() method?</span><br /> <span class="token keyword">const</span> actual <span class="token operator">=</span> <span class="token function">getBatchSize</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">expect</span><span class="token punctuation">(</span>actual<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toEqual</span><span class="token punctuation">(</span>testVal<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>But this is where I've run into my first hurdle. How do I implement this in a way that is both (1) Testable, and (2) not cumbersome to the application? Sure, I can take every method that I would ever want to write and make it into its own module so that I can test it in isolation, but I can see that very quickly getting to insanity-inspiring numbers of files. There must be a better way!</p>
<p>So, let me give two examples to illustrate the two concepts that I feel are at odds with one another.</p>
<p>First, here's how I might write the code to cleanly get the batch size in a reusable manner, if I wasn't at all concerned with testing:</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// lib/redis-click-logger.js</span><br />module<span class="token punctuation">.</span><span class="token function-variable function">exports</span> <span class="token operator">=</span> <span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> batchSize <span class="token operator">=</span> <span class="token function">getBatchSize</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token comment">// ...</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">function</span> <span class="token function">getBatchSize</span><span class="token punctuation">(</span><span class="token parameter">defaultBatchSize <span class="token operator">=</span> <span class="token number">1000</span></span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> process<span class="token punctuation">.</span>env<span class="token punctuation">.</span><span class="token constant">BATCH_SIZE</span> <span class="token operator">||</span> defaultBatchSize<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>The problem with this approach is that it doesn't export the <code>getBatchSize</code> function, and as far as I know, there's no good way to test that function in isolation as-written.</p>
<p>I come from a CFML background, and this is how we might write the same module in CFML:</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// lib/redis-click-logger.cfc</span><br />component <span class="token punctuation">{</span><br /><br /> <span class="token keyword">public</span> <span class="token keyword">function</span> <span class="token function">processClickBatch</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">var</span> batchSize <span class="token operator">=</span> <span class="token function">getBatchSize</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">getBatchSize</span><span class="token punctuation">(</span> <span class="token parameter">defaultBatchSize <span class="token operator">=</span> <span class="token number">1000</span></span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> server<span class="token punctuation">.</span>system<span class="token punctuation">.</span>environment<span class="token punctuation">[</span><span class="token string">'BATCH_SIZE'</span><span class="token punctuation">]</span> <span class="token operator">?</span><span class="token operator">:</span> arguments<span class="token punctuation">.</span>defaultBatchSize<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>In the above example, the <code>processClickBatch()</code> method is what's initially called upon to do the work, and it has access to call the private methods of the module. The testing and mocking tools available to us in CFML make it trivial to make the <code>getBatchSize()</code> function public during the test, and then it pretty much goes how I outlined in the test implementation I wrote above. Set some arbitrary value, run the getter, assert that it returned what you expected.</p>
<p>But how do I accomplish this in JavaScript? As previously mentioned, the only idea I've been struck with so far was to externalize all methods as modules and require them — which sounds like a patently bad idea.</p>
<p>Of course JavaScript has classes, but I hate them. More than just hate, they are not the recommended pattern for React.js apps, of which we have several and expect that we'll be writing a TON of React code in the near future. That code will need testing too, so even if classes could solve my problems now, I expect I'll run into the same problems later when we're using functional components and hooks, right? All the more reason to avoid JS classes as a solution now.</p>
<p>I feel like there's some 3rd option that I'm not seeing. There's gotta be, right?</p>
<p>Now here's where reality is setting in: I pretty much burned my entire day yesterday trying to get over this hurdle. Testing is important, but I can't let something so trivial completely block me from getting any work done. So for now, I'm putting TDD to the back of the stove and when I understand how to solve these problems I will come back and refactor my code to be testable and write the tests. In the meantime, I've got to get some work done.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>By the by: If you're looking for a fantastic community of JavaScript developers, the <a href="https://kentcdodds.com/discord">KCD Discord</a> is amazing. I can't recommend it strongly enough. <a href="https://adamtuttle.codes/blog/2021/lip-writing-testable-nodejs-code/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Testing Untestable CFML2021-05-25T00:00:00Zhttps://adamtuttle.codes/blog/2021/testing-untestable-cfml/<p>I spoke at some length about my frustrations testing CFML on the <a href="https://workingcode.dev/episodes/009-testing/">"Testing" episode of my podcast, Working Code</a> (in February), and I've had some realizations since then, so I wanted to share what I've learned.</p>
<p><strong>Without a doubt, it was me that failed. It is 100% possible to write clean, testable CFML code, and performant tests.</strong></p>
<p>In fact, something I didn't understand about TDD until recently was that its primary benefit —at least in my opinion— is not that you've written the tests before/with the code, but rather that <em>it forces you to write testable code.</em></p>
<p>This pattern could be repeated with nearly any feature, but the thing I had in mind when I lamented the inability to test my CFML app quickly and efficiently was ORM. Specifically, Adobe's implementation of Hibernate. What I was feeling was that it was impossible to test code that used ORM, but that feeling was wrong. If I had known enough to continue scratching that itch, I might have figured this out sooner.</p>
<p>And I do mean that I figured it out. Even though I have a bachelors degree in Computer Science and roughly 20 years of full time professional coding experience, somehow I managed to make it this far without ever getting a truly ground-up testing education. And what that's meant is that I had to do things the wrong way for a long time, and learn what it feels like to have those pains; <em>AND</em> I had to be exposed to the solutions multiple times because I never truly understood them the at the first few opportunities.</p>
<p>Eventually, through brute force, understanding starts to set in. So here it is. Here's the lesson that I want you to take away from this article:</p>
<p><strong>If there's something that feels like it's in the way of testing your code, THAT is what's making your code unclean.</strong></p>
<p>ORM usage is not inherently unclean. But the way that I was using it definitely was, and the design choices I was making seemed innocuous enough at the time...</p>
<p>Why wouldn't the service layer just go ahead and use ORM entities to load data from the database, or make data mutations? That's a very app-logic thing, right? The data is kind of part of the app.</p>
<p><em>... right?</em></p>
<p>To be clear, even if I had been using SQL instead of ORM, I would have had the exact same problem. My problem wasn't that I was using ORM, it was that Data Access was being intermingled with application logic.</p>
<h2 id="data-access-objects" tabindex="-1">Data Access Objects</h2>
<p>Some time around the year 2008 I recall using a tool from <a href="https://remotesynthesis.com/">Brian Rinaldi</a> that would inspect your database and generate a bunch of code to save you time. It output models and services and data access objects. And if I'm being honest, while I thought that I "got it" at the time, I definitely did not.</p>
<p>Data Access Objects exist not just for separation of concerns (keeping db mutations away from app logic) but also to make mocking them <s>easy</s> possible.</p>
<p>Going back to my somewhat-recent frustrations testing CFML, one of my biggest struggles was that for every test I had to make sure that my test-db had its data in the appropriate state before test, then I had to run the test, then I had to make sure that the data in the db was in the state that I expected after the test. Repeat that a few thousand times and it starts to become painfully obvious why you should be mocking the data access layer during tests.</p>
<p>So, if I understand <s>correctly</s> better now, the DAO layer should contain all database access. All readers and all writers. And, more to the point, it doesn't matter if that DAO layer uses ORM or queries or a mix of both or something else.</p>
<h2 id="testing-untestable-cfml" tabindex="-1">Testing Untestable CFML</h2>
<p>So sure, the "untestable" bit of the title may be a little click-bait-y but how many of us have <em>at least one</em> app that we consider "untestable"? I'd wager a lot more than 50%!</p>
<p>So here's how you test it: You find the things that make it "untestable," and you pick at those scabs until you understand what it would take to make it testable. Then you refactor the code to make it testable, and you write tests.</p>
<p>Some would say that if you don't have a test for the code you're changing then "<a href="http://hamletdarcy.blogspot.com/2009/06/forgotten-refactorings.html">you're not refactoring, you're just changing shit.</a>" I'm starting to agree. Though, if you have existing code and you need to refactor it in order to make it testable and write tests, I don't suppose you have much choice, do you?</p>
<h2 id="an-example" tabindex="-1">An Example</h2>
<p>Say we have a <a href="https://framework-one.github.io/">FW/1</a> service for blog posts, with a save method that you use to save changes when the create/edit form is submitted:</p>
<pre class="language-js"><code class="language-js">component<br />accessors<span class="token operator">=</span><span class="token boolean">true</span><br /><span class="token punctuation">{</span><br /><br /> <span class="token keyword">function</span> <span class="token function">save</span> <span class="token punctuation">(</span> <span class="token parameter">id<span class="token punctuation">,</span> author<span class="token punctuation">,</span> title<span class="token punctuation">,</span> body<span class="token punctuation">,</span> tags</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//option 1: write via query</span><br /> <span class="token function">queryExecute</span><span class="token punctuation">(</span> <span class="token comment">/* create/update the post */</span> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token comment">//option 2: write via ORM</span><br /> transaction <span class="token punctuation">{</span><br /> <span class="token keyword">var</span> post <span class="token operator">=</span> <span class="token function">entityLoadByPk</span><span class="token punctuation">(</span> <span class="token string">'post'</span><span class="token punctuation">,</span> arguments<span class="token punctuation">.</span>id <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> post<span class="token punctuation">.</span><span class="token function">setTitle</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>title<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token comment">//... etc</span><br /> <span class="token function">entitySave</span><span class="token punctuation">(</span> post <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>How do you write tests for this service?</p>
<p>The biggest problem depicted here is that the service's save method directly updates the database. As shown, you could be doing this via SQL queries or ORM and have the same problem. How do you mock the database operations to make the tests faster and avoid the actual-database-setup-and-teardown work?</p>
<p>It should be obvious now, right? Split it out into a Data Access Object ("DAO"), and then mock that DAO in your tests.</p>
<pre class="language-js"><code class="language-js"><span class="token comment">// postService</span><br />component<br />accessors<span class="token operator">=</span><span class="token boolean">true</span><br /><span class="token punctuation">{</span><br /><br /> property postDAO<span class="token punctuation">;</span><br /><br /> <span class="token keyword">function</span> <span class="token function">save</span> <span class="token punctuation">(</span> <span class="token parameter">id<span class="token punctuation">,</span> author<span class="token punctuation">,</span> title<span class="token punctuation">,</span> body<span class="token punctuation">,</span> tags</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> postDAO<span class="token punctuation">.</span><span class="token function">upsert</span><span class="token punctuation">(</span> id<span class="token punctuation">,</span> author<span class="token punctuation">,</span> title<span class="token punctuation">,</span> slug<span class="token punctuation">,</span> body<span class="token punctuation">,</span> tags <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span><br /><br /><span class="token comment">// postDAO</span><br />component <span class="token punctuation">{</span><br /><br /> <span class="token keyword">function</span> <span class="token function">upsert</span><span class="token punctuation">(</span> <span class="token parameter">id<span class="token punctuation">,</span> author<span class="token punctuation">,</span> title<span class="token punctuation">,</span> slug<span class="token punctuation">,</span> body<span class="token punctuation">,</span> tags</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//update the database accordingly</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>Now, when you write your tests for <code>postService</code>, you can inject the appropriate mock for the test that you're running, the service will have no idea that it's not actually talking to a database, the tests will run much faster, and you won't have to do slow and tedious actual-db setup and teardown.</p>
<p>I'm not going to show how to do the mocking details, just enough to paint the picture of what your tests should look like:</p>
<pre class="language-js"><code class="language-js"><span class="token function">describe</span><span class="token punctuation">(</span><span class="token string">'postService'</span><span class="token punctuation">,</span> <span class="token keyword">function</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">it</span><span class="token punctuation">(</span><span class="token string">'writes to the database'</span><span class="token punctuation">,</span> <span class="token keyword">function</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token comment">//this next line is mostly just hand-waving...</span><br /> <span class="token keyword">var</span> postDAOMock <span class="token operator">=</span> <span class="token function">createPostDAOMock</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">var</span> postService <span class="token operator">=</span> beanFactory<span class="token punctuation">.</span><span class="token function">getBean</span><span class="token punctuation">(</span><span class="token string">'postService'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> postService<span class="token punctuation">.</span><span class="token function">setPostDAO</span><span class="token punctuation">(</span>postDAOMock<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> postService<span class="token punctuation">.</span><span class="token function">save</span><span class="token punctuation">(</span><span class="token comment">/* ... */</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token comment">// use expectations to assert that postDAOMock.upsert was called</span><br /> <span class="token comment">// with certain arguments, or other valuable assertions</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>As I alluded to in a recent twitter thread, I wasn't raised in a house that made flossing a priority, and likewise many of my early jobs placed near-zero priority on automated testing, and I've found a bunch of parallels between the two.</p>
<p><a href="https://twitter.com/AdamTuttle/status/1395750563078365188">https://twitter.com/AdamTuttle/status/1395750563078365188</a></p>
<p>Obviously I still have a lot to learn in the world of testing. I'll be sure to continue sharing what I learn here.</p>
Tabs vs. Spaces: It's an Accessibility Issue2021-05-24T00:00:00Zhttps://adamtuttle.codes/blog/2021/tabs-vs-spaces-its-an-accessibility-issue/<p>I don't remember where I got my preference for Tabs over Spaces, but I've been ride-or-die Tabs for as long as I can remember. I wasn't joking when I <a href="https://adamtuttle.codes/blog/2021/sweat-the-small-stuff/">included that "tabs rule, spaces drool" meme in my "Sweat the Small Stuff" article</a>. I run in Team Tabs and while I do try to have "strong opinions weakly held," I am skeptical that anyone would ever be able to change my mind.</p>
<p><img src="https://adamtuttle.codes/img/2021/tabs-rule.jpg" alt="Once again for the people in the back: Tabs rule, spaces drool. Change my mind." /></p>
<p>This came up in the Aftershow discussion for <a href="https://workingcode.dev/">my podcast</a> many weeks ago, and my friend and co-host <a href="https://www.bennadel.com/">Ben Nadel</a> described it as an "accessibility issue," which is just such a perfect description...</p>
<div style="max-width: 37.5em;">
<div class="tenor-gif-embed" data-postid="16446354" data-share-method="host" data-width="100%" data-aspect-ratio="1.7785714285714287"><a href="https://tenor.com/view/john-stewart-the-jon-stewart-show-kiss-chef-kiss-perfetto-gif-16446354">John Stewart The Jon Stewart Show GIF</a> from <a href="https://tenor.com/search/johnstewart-gifs">Johnstewart GIFs</a></div><script type="text/javascript" async="" src="https://tenor.com/embed.js"></script>
</div>
<p>The argument goes like this: People are welcome to have their own opinions on how much indentation they would like to see. Perhaps you like 2-character-width indentation, or 8, or somewhere in-between. By using a literal space character for indentation, you are forcing your preference on others who read that code.</p>
<p>Now contrast that with tabs. 1 tab character is 1 level of indentation. If my preference is 2-character-width indentation, that's what I have set in my editor and I see space equivalent to 2 characters. And my teammate who prefers 6 or 8 characters can see it the way they want it.</p>
<p>Now, that's all fine, but here's where it becomes an accessibility issue.</p>
<p>People with less than perfect eyesight can have trouble differentiating indentation when the tab-width is low. For accessibility reasons, we need to be able to see more space. And the more code we're looking at, the more it's needed. Having to scroll down 14 lines of code and keep 6 levels of indentation aligned by eye is so much easier with a wider tab-width.</p>
<p>Nothing against those of you with fantastic vision —I used to be one of you— but when you can get your preference in a way that allows me to also be able to read and understand the code, that is <strong>objectively better</strong> than an alternative that only lets one of us get what we want or need.</p>
<p>Tabs are just better. It's an accessibility issue.</p>
<h2 id="addendum" tabindex="-1">Addendum</h2>
<p>Occasionally I find even more slam-dunk evidence that tabs is the only correct way to indent code, and I've decided that when I do I'm going to add it here.</p>
<p>Thanks to <a href="https://twitter.com/Rich_Harris/status/1541761871585464323">Rich Harris</a> for pointing this one out:</p>
<blockquote>
<p>The main reason I would like to see this change is for refreshable braille displays that are used by blind programmers a lot. Each space wastes one braille cell and takes away valuable braille realestate. So if the default indentation of a project is 4 spaces per level, a 3rd level indentation wastes 12 braaille cells before code starts. On a 40 cell display, which is the most commonly used with notebooks, this is more than a quarter of the available cells wasted with no information. If each indentation level was represented by only one tab character, there would be three cells occupied by a tab character each, and on the 4th cell, the code would start. That's less than 10 percent occupied on the same length display, but all cells contain valuable information that is easily discoverable and immediately comprehensible.</p>
<p>— <a href="https://github.com/prettier/prettier/issues/7475#issuecomment-668544890">MarcoZehe on GitHub</a> in a PR requesting that prettier change their <code>useTabs</code> setting from default <code>false</code> to default <code>true</code>.</p>
</blockquote>
How Semaphore Calculates Percentage-Based Rollouts2021-05-21T00:00:00Zhttps://adamtuttle.codes/blog/2021/how-semaphore-calculates-percentage-based-rollouts/<p>When I first started learning about feature flags, I found it all somewhat interesting <strong>but predictable.</strong> So while I was excited about the prospect of using this new tool, it didn't pose any interesting challenges. That was... until I heard about percentage based rollouts. This was a problem that didn't seem (to me, at least) to have an obvious solution, and so I couldn't stop thinking about it.</p>
<p>I imagined different possible implementations, and found flaws in them all. How was it decided which users would get it? Was it the first N% of users that got to the feature? Surely there's an element of randomness to it so that you don't choose the same people every time you roll a flag out to 20% of users, right?</p>
<p>If you listen to <a href="https://workingcode.dev/episodes/018-feature-flags-finally/">my podcast</a>, then you've probably heard this already, but the short version is that basically you get some inputs that represent the user, and then you deterministically convert that into a number that you can represent as a percentage (e.g. 0-100 or 0-1), and if the user's percentage is <= the rollout percentage, then the user has that flag turned on.</p>
<p>It sounds simple, but it's kinda not. How do you take an arbitrary chunk of data and deterministically convert it into a number?</p>
<p>I'm not aware of any CFML or Java methods that do this for you. Actually, that's a lie. I am, but once again I'm thwarted by differences in CFML engines. Java's <code>string.hashCode()</code> method does return an integer value, but is not usefully implemented across all CFML engines. Still, even if that were an option, getting from an object to a string in a deterministic manner is, well, <a href="https://adamtuttle.codes/blog/2021/chaotic-good-creating-determinism-where-none-exists/">not a straight-forward problem to solve in CFML</a>.</p>
<p>One thing that I alluded to in that entry was that I'm including both the <code>userAttributes</code> and the flag rule in my data 👉 number calculation. It's not strictly necessary in order to get a number that represents the user; but if you don't then it doesn't solve for the "same users selected every time" problem. By adding the flag into the hashing algorithm, it introduces an element of randomness seeded on the flag itself, which is exactly what we need, so that user selection is randomized between all percentage-based flags.</p>
<p>Still, I was curious how well my algorithm distributes values across the range of possible values, so I did the math.</p>
<p>Does my algorithm (somewhat) evenly distribute those numbers across the range of possible values? (click for full size)</p>
<p><a href="https://adamtuttle.codes/img/2021/md5-random1.png"><img src="https://adamtuttle.codes/img/2021/md5-random1.png" alt="bar chart showing the numbers 1-100 on the x-axis, and a random but fairly even distribution of approximately 1,000 in the y-axis for each bar" /></a></p>
<p>It sure seems like it. So how did I come up with those numbers?</p>
<p>The goal is to capture "CRC" values (apologies, I'm using the term CRC extremely loosely -- if you've got a better term in mind, please leave it in a comment!) for a series of inputs that are meant to represent unique users, each different from the last. In order to not simply be calculating the MD5 of an incrementing counter, I introduced a random string generator to produce a "name" for each <code>userData</code> object, too. And they were all tested against the same "rule".</p>
<pre class="language-js"><code class="language-js"><span class="token operator"><</span>cfscript<span class="token operator">></span><br /> semaphore <span class="token operator">=</span> <span class="token function">createObject</span><span class="token punctuation">(</span><span class="token string">'lib.semaphore'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> attempts <span class="token operator">=</span> url<span class="token punctuation">.</span><span class="token function">keyExists</span><span class="token punctuation">(</span><span class="token string">'attempts'</span><span class="token punctuation">)</span> <span class="token operator">?</span> url<span class="token punctuation">.</span>attempts <span class="token operator">:</span> <span class="token number">1000</span><span class="token punctuation">;</span><br /><br /> distribution <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><=</span> <span class="token number">100</span><span class="token punctuation">;</span> i<span class="token operator">++</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> distribution<span class="token punctuation">[</span>i<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">;</span> i <span class="token operator"><=</span> attempts<span class="token punctuation">;</span> i<span class="token operator">++</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> userData <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token literal-property property">name</span><span class="token operator">:</span> <span class="token function">RandomString</span><span class="token punctuation">(</span><span class="token number">15</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token literal-property property">i</span><span class="token operator">:</span> i <span class="token punctuation">}</span><span class="token punctuation">;</span><br /> rule <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token literal-property property">testing</span><span class="token operator">:</span> <span class="token string">'yo'</span> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /> result <span class="token operator">=</span> semaphore<span class="token punctuation">.</span><span class="token function">getUserRuleCRC</span><span class="token punctuation">(</span> userData<span class="token punctuation">,</span> rule <span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token function">storeResult</span><span class="token punctuation">(</span> result <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">function</span> <span class="token function">storeResult</span><span class="token punctuation">(</span> <span class="token parameter">required numeric result</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//because of the way percentages are checked in semaphore with <=,</span><br /> <span class="token comment">//using ceiling ensures we're measuring apples to apples.</span><br /> place <span class="token operator">=</span> <span class="token function">ceiling</span><span class="token punctuation">(</span>result <span class="token operator">*</span> <span class="token number">100</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> distribution<span class="token punctuation">[</span>place<span class="token punctuation">]</span><span class="token operator">++</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token comment">//https://lunaticthinker.me/index.php/cfml-generating-random-string/</span><br /> <span class="token keyword">function</span> <span class="token function">RandomString</span><span class="token punctuation">(</span>length<span class="token punctuation">,</span> chars<span class="token operator">=</span><span class="token string">"ABCDEFGHIJKLMNOPQRST0123456789-"</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> rs <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span><br /> theString <span class="token operator">=</span> <span class="token string">""</span><span class="token punctuation">;</span><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span>rs <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> rs <span class="token operator"><</span> arguments<span class="token punctuation">.</span>length<span class="token punctuation">;</span> rs<span class="token operator">++</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> theString <span class="token operator">&=</span> <span class="token function">Mid</span><span class="token punctuation">(</span>chars<span class="token punctuation">,</span> <span class="token function">RandRange</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token function">len</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>chars<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">"SHA1PRNG"</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">return</span> theString<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token operator"><</span><span class="token operator">/</span>cfscript<span class="token operator">></span><br /><br /><span class="token operator"><</span>cfchart format<span class="token operator">=</span><span class="token string">"png"</span> title<span class="token operator">=</span><span class="token string">"CRC distribution"</span> chartheight<span class="token operator">=</span><span class="token string">"500"</span> chartwidth<span class="token operator">=</span><span class="token string">"1800"</span><span class="token operator">></span><br /> <span class="token operator"><</span>cfchartseries type<span class="token operator">=</span><span class="token string">"bar"</span><span class="token operator">></span><br /> <span class="token operator"><</span>cfloop from<span class="token operator">=</span><span class="token string">"0"</span> to<span class="token operator">=</span><span class="token string">"100"</span> index<span class="token operator">=</span><span class="token string">"d"</span><span class="token operator">></span><br /> <span class="token operator"><</span>cfchartdata item<span class="token operator">=</span><span class="token string">"#d#"</span> value<span class="token operator">=</span><span class="token string">"#distribution[d]#"</span> <span class="token operator">/</span><span class="token operator">></span><br /> <span class="token operator"><</span><span class="token operator">/</span>cfloop<span class="token operator">></span><br /> <span class="token operator"><</span><span class="token operator">/</span>cfchartseries<span class="token operator">></span><br /><span class="token operator"><</span><span class="token operator">/</span>cfchart<span class="token operator">></span></code></pre>
<p>A note, if you care to run this code yourself: <code>semaphore.getUserRuleCRC</code> is a private method. I thought about including a public proxy or doing some work to use mixins to make it public as part of the above script, but that all seemed like a great waste of time when I can just toggle the function to public when I want to run the script. So you'll need to do the same.</p>
<p>You may have noticed that the chart I showed above has the <a href="https://en.wikipedia.org/wiki/Law_of_large_numbers">Law of Large Numbers</a> on its side, since I ran it for a set of 100,000 inputs. Indeed, if I run it for a million inputs the top of the chart looks even flatter. So let's have a quick peek at the results for 1,000 inputs instead: (click for full size)</p>
<p><a href="https://adamtuttle.codes/img/2021/md5-random2.png"><img src="https://adamtuttle.codes/img/2021/md5-random2.png" alt="bar chart showing the numbers 1-100 on the x-axis, and a random but fairly even distribution of approximately 1,000 in the y-axis for each bar" /></a></p>
<p>That is certainly not as evenly distributed, but honestly it's better than I expected for such a ridiculous algorithm as what I've done, and definitely useful for segmenting users. You may not get exactly 37% when you specify 37%, but you'll get much closer to 37% than you will to 0% or 100%, and at the end of the day, I think that's what really matters.</p>
<p>Maybe it's not such an insane algorithm after all?</p>
Publishing Flag Changes with Semaphore2021-05-20T00:00:00Zhttps://adamtuttle.codes/blog/2021/publishing-flag-changes-with-semaphore/<p>In <a href="https://adamtuttle.codes/blog/2021/introducing-semaphore/">yesterday's announcement</a> about the release of <a href="https://github.com/atuttle/semaphore">Semaphore</a>, I wrote a little bit about how I'll be wrapping Semaphore with a service which makes it easy to evaluate flags anywhere in my application, but one thing I didn't describe was my plan for publishing new/updated flags to each application instance as they change. So let's do that.</p>
<p>One of the primary benefits of feature flags is the ability to decouple deploy from release, and in order to accomplish that decoupling you need to have a mechanism for updating the local flag state cache. You could set up a scheduled task to poll your persistence layer for any updates, but the frequency with which I expect flags to be updated (pretty low, maybe a few times per day) is at odds with the speed at which I want my flag changes to be running in production. No, a push mechanism is definitely needed.</p>
<p>In the off-the-shelf projects like LaunchDarkly, the impression that I get is that their SDK creates an in-memory cache that loads at app start and that they can push changes into when you make them from their admin interface. I want something like that.</p>
<p>There's not much to it, really. The flag data gets updated in persistence (Redis, in my case), and then I'll notify each server to fetch the latest flags on-demand. This way I don't have to have dozens of servers all DDoSing my Redis cluster every 15 seconds to check for flag updates, but also my changes will deploy ~immediately. Best of both worlds.</p>
<p>My service has a method for updating the local flag cache:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token keyword">function</span> <span class="token function">updateFeatureFlagCache</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">var</span> flags <span class="token operator">=</span> redisService<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">'feature-flags'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> semaphore<span class="token punctuation">.</span><span class="token function">setAllFlags</span><span class="token punctuation">(</span> flags <span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Plop this behind a publicly-addressable but moderately-secret URL and you've got a realtime-ish publish mechanism for flag changes without an application deploy. Whatever interface you use to manage your flags should loop over the list of applicable servers and notify them each to update their cache any time a flag is updated.</p>
<p>That's it. It's really that simple. Even I thought there might be more to explore here, but nope.</p>
Introducing Semaphore2021-05-19T00:00:00Zhttps://adamtuttle.codes/blog/2021/introducing-semaphore/<p>Today I'm excited to tell you about a new open source project I've been working on, called <a href="https://github.com/atuttle/semaphore">Semaphore</a>.<sup>1</sup></p>
<p>It's a minimalist feature flags engine for CFML applications. What's <strong>not</strong> included? Persistence of flag data, and a UI for flag management.</p>
<p>If you're not familiar with what <strong>feature flags</strong> are, I would encourage you to listen to <a href="https://workingcode.dev/episodes/018-feature-flags-finally/">Working Code episode 18: "Feature Flags (Finally!)"</a>, or read <a href="https://www.bennadel.com/blog/3766-my-personal-best-practices-for-using-launchdarkly-feature-flags.htm">Ben Nadel's article explaining how his team uses them</a>. Or both.</p>
<h2 id="why-would-i-implement-my-own-engine%3F" tabindex="-1">Why would I implement my own engine?</h2>
<p>Aside from the obvious, "Because I can, and it was fun!" answer?</p>
<p>Why not use an existing service? Well, <a href="https://launchdarkly.com/">LaunchDarkly</a> is out of the price range for the balance of what I want to do with it and what I can afford, so that rules it out. I also found <a href="https://flagsmith.com/">FlagSmith</a> and <a href="https://www.split.io/">Split.io</a>, and both have Java SDK's, so I tried using both of them. I was unsuccessful with both of them.</p>
<p>I'm no idiot, but I'm also not a seasoned Java developer. And while CFML is a layer on top of Java, that doesn't mean documentation for Java developers makes sense to me or even correctly explains how you would use the library in CFML. I'm unfamiliar with Maven, which both seem to recommend, and from what I've heard it's not a particularly friendly tool either. When all was said and done I had spent 3-4 evenings of my free time and all I had to show for it was that I proved I wasn't capable of getting either SDK to even load into the most basic usable state in a CFML app.</p>
<p>Since I'm not willing to give up on feature flags, I decided I had one recourse left: roll my own.</p>
<h2 id="so-what-does-it-do%3F" tabindex="-1">So what <em>does</em> it do?</h2>
<p>I told you what it <em>doesn't</em> do (to recap: persistence, UI), so what <em>does</em> it do?</p>
<ol>
<li>Define a <acronym title="Domain Specific Language">DSL</acronym> for defining flags with data primitives (more on that later)</li>
<li>Provide <acronym title="Create, Read, Update, Delete">CRUD</acronym> methods for managing flags in memory</li>
<li>And provide a method for evaluating whether a given user has the flag turned ON or OFF based on the rules defined in the flags</li>
</ol>
<p>I can see how a provided UI might be beneficial to people who run a single instance of their application on a single server, but I don't want artificially limit Semaphore's usefulness by making any design decisions for that mental model. That said, it leaves a nice opening if someone wants to develop a companion project. 😉</p>
<h2 id="how-i'll-be-using-it" tabindex="-1">How I'll be using it</h2>
<p>Instead, I intend to use it in my application by calling it from a service that has the responsibility of loading the flags from my persistence mechanism, and proxying flag-check calls; and writing a separate small application for viewing the state of my flags and managing them in the persistence store. We'll be using Redis, because it's fast, lightweight, and because all of my application instances are already using it for other reasons, so why not?</p>
<p>This should make it easily useful for a multi-tenant, multi-instance, distributed application, which is important to me.</p>
<h2 id="defining-flags" tabindex="-1">Defining flags</h2>
<p>Here's how you define flags:</p>
<pre class="language-js"><code class="language-js">semaphore<span class="token punctuation">.</span><span class="token function">setAllFlags</span><span class="token punctuation">(</span><span class="token punctuation">{</span><br /> <span class="token literal-property property">myFeature</span><span class="token operator">:</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">name</span><span class="token operator">:</span> <span class="token string">'Flag Name'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">description</span><span class="token operator">:</span> <span class="token string">'A brief description of what the flag is used for'</span><span class="token punctuation">,</span><br /><br /> <span class="token comment">// if a flag is inactive, it will always return false</span><br /> <span class="token literal-property property">active</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /><br /> <span class="token comment">// optional. default: false. set to true to invert the response</span><br /> <span class="token comment">// => true becomes false, false becomes true.</span><br /> <span class="token literal-property property">baseState</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span><br /><br /> <span class="token comment">// The flag will be ON for any user who matches AT LEAST ONE rule</span><br /> <span class="token literal-property property">rules</span><span class="token operator">:</span> <span class="token punctuation">[</span><br /> <span class="token punctuation">{</span><br /> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">'%'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">percentage</span><span class="token operator">:</span> <span class="token number">42</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">{</span><br /> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">'filter'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">attribute</span><span class="token operator">:</span> <span class="token string">'userId'</span><span class="token punctuation">,</span><br /> <span class="token comment">// supported math operators: ==, !=, <, <=, >, >=</span><br /> <span class="token literal-property property">operator</span><span class="token operator">:</span> <span class="token string">'<='</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">comparator</span><span class="token operator">:</span> <span class="token number">42</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">{</span><br /> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">'filter'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">attribute</span><span class="token operator">:</span> <span class="token string">'role'</span><span class="token punctuation">,</span><br /> <span class="token comment">// use "has" operator to find comparator in attribute array</span><br /> <span class="token literal-property property">operator</span><span class="token operator">:</span> <span class="token string">'has'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">comparator</span><span class="token operator">:</span> <span class="token string">'beta-tester'</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">{</span><br /> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">'filter'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">attribute</span><span class="token operator">:</span> <span class="token string">'cohort'</span><span class="token punctuation">,</span><br /> <span class="token comment">// use "in" operator to find attribute value in comparator array</span><br /> <span class="token literal-property property">operator</span><span class="token operator">:</span> <span class="token string">'in'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">comparator</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token number">5</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">,</span> <span class="token number">9</span><span class="token punctuation">]</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">{</span><br /> <span class="token comment">// ON for everybody</span><br /> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">'everybody'</span><br /> <span class="token punctuation">}</span><span class="token punctuation">,</span><br /> <span class="token punctuation">{</span><br /> <span class="token comment">// OFF for everybody</span><br /> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">'nobody'</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">]</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>At present, the examples above show all supported rule types. Of course, you should include only the rules you want to apply to your flag.</p>
<p>As the method name shown implies, this method overwrites the entire set of flags with the data you provide. This is useful for bulk-creation at app startup. These methods are all provided and should hopefully be pretty self-explanatory:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">void</span> <span class="token function">setAllFlags</span><span class="token punctuation">(</span>required struct flags<span class="token punctuation">)</span><br /><br />struct <span class="token function">getAllFlags</span><span class="token punctuation">(</span><span class="token punctuation">)</span><br /><br /><span class="token keyword">void</span> <span class="token function">setFlag</span><span class="token punctuation">(</span>required string flagId<span class="token punctuation">,</span> required struct flag<span class="token punctuation">)</span><br /><br />struct <span class="token function">getFlag</span><span class="token punctuation">(</span>required string flagId<span class="token punctuation">)</span></code></pre>
<h2 id="evaluating-flags" tabindex="-1">Evaluating flags</h2>
<p>Once you have a Semaphore instance on its feet, the next step is to start putting features behind flags. As I mentioned, I'll be wrapping Semaphore in a service because that makes it easy to inject anywhere in my application that I might need it using <acronym title="Dependency Injection">DI</acronym>. It also has the added benefit that my service wrapper can abstract the inclusion of user attributes in flag checks so that I don't have to include it in every flag location.</p>
<p>Here's what Semaphore is expecting you to call to check a flag status:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">if</span> <span class="token punctuation">(</span><br /> semaphore<span class="token punctuation">.</span><span class="token function">checkForUser</span><span class="token punctuation">(</span><br /> <span class="token literal-property property">flagId</span><span class="token operator">:</span> <span class="token string">'myFeature'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">userAttributes</span><span class="token operator">:</span> <span class="token punctuation">{</span><br /> <span class="token comment">// the data here is not significant</span><br /> <span class="token comment">// it's included only to give you an idea of what you might want</span><br /> <span class="token literal-property property">userId</span><span class="token operator">:</span> <span class="token number">42</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">roles</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">'beta-tester'</span><span class="token punctuation">,</span><span class="token string">'admin'</span><span class="token punctuation">,</span><span class="token string">'plebe'</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> <span class="token operator">...</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">)</span><br /><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token function">theNewImplementation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token function">theOldImplementation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>The value for userAttributes should be the same for each user every time they login, assuming the underlying data hasn't changed; so you should probably calculate that as part of login and then cache it somewhere like the user session. That simplifies it a little bit:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">if</span> <span class="token punctuation">(</span><br /> semaphore<span class="token punctuation">.</span><span class="token function">checkForUser</span><span class="token punctuation">(</span><br /> <span class="token literal-property property">flagId</span><span class="token operator">:</span> <span class="token string">'myFeature'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">userAttributes</span><span class="token operator">:</span> session<span class="token punctuation">.</span>userAttributes<br /> <span class="token punctuation">)</span><br /><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token function">theNewImplementation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token function">theOldImplementation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>By wrapping it in a service, I can have the service proxy the call to checkForUser and take over responsibility of including the user attributes argument:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">if</span> <span class="token punctuation">(</span>featureFlagService<span class="token punctuation">.</span><span class="token function">flagIsOn</span><span class="token punctuation">(</span><span class="token string">'myFeature'</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token function">theNewImplementation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token function">theOldImplementation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p><em><strong>Nice.</strong></em></p>
<p>Semaphore is <a href="https://github.com/atuttle/semaphore">available on GitHub</a> and near as I can tell it's feature-complete and well tested. I haven't had time to use it in my application YET, so I can't vouch for it. But... SOON. Thanks to Ben and the podcast I am pretty eager to give it a go.</p>
<hr />
<p><small><sup>1</sup>This is, by the way, the project I was working on when I wrote my recent entry, <a href="https://adamtuttle.codes/blog/2021/chaotic-good-creating-determinism-where-none-exists/">Chaotic Good: Creating Determinism Where None Exists</a>.</small></p>
Chaotic Good: Creating Determinism Where None Exists2021-05-14T00:00:00Zhttps://adamtuttle.codes/blog/2021/chaotic-good-creating-determinism-where-none-exists/<p>I was having a chat with <a href="https://blog.adamcameron.me/">Adam Cameron</a> yesterday and bemoaning how, ahem, <em>certain CFML engines</em> do not serialize Structs in a deterministic way. After some back and forth, he hit me with an idea so good that I am still reeling from it the next day. I sang his praises last night while recording a new episode of <a href="https://workingcode.dev/">Working Code</a> (that episode will publish on 5/26), and yet I still feel the need to shout his brilliance from the mountaintops more. So...</p>
<p>Firstly, what is determinism? It's not a concept unique to programming, but it matters a lot to us. So let's look at it from a programmer's perspective.</p>
<h2 id="what-is-determinism%3F" tabindex="-1">What is determinism?</h2>
<p>A function can be said to be <strong>deterministic</strong> if its output can be predicted using only its input (e.g. state of the system has no effect), and repeating the same inputs always produces the same outputs. This function is deterministic:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token parameter">a<span class="token punctuation">,</span> b</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> a <span class="token operator">+</span> b<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>However, this function is NOT deterministic:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">maths</span><span class="token punctuation">(</span><span class="token parameter">a<span class="token punctuation">,</span> b</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> a <span class="token operator">+</span> b <span class="token operator">+</span> mouse<span class="token punctuation">.</span>x <span class="token operator">*</span> mouse<span class="token punctuation">.</span>y<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>It would be easy to confuse determinism and pure functions, so while we're on the subject, what makes them different? A pure function is a deterministic function that also <em>produces no side-effects</em>. This function is deterministic but not pure:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token parameter">a<span class="token punctuation">,</span> b</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> stdout<span class="token punctuation">.</span><span class="token function">print</span><span class="token punctuation">(</span>a <span class="token operator">+</span> b<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> a <span class="token operator">+</span> b<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Ok, so with that now understood, what did I mean about some CFML engines not having a deterministic implementation of <code>serializeJson()</code>?</p>
<h2 id="when-is-serializejson-non-deterministic%3F" tabindex="-1">When is serializeJson non-deterministic?</h2>
<p>Given a struct created using <em>ANY</em> method; not necessarily the implicit object notation shown here:</p>
<pre class="language-js"><code class="language-js">data <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token constant">A</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span><br /> <span class="token constant">B</span><span class="token operator">:</span> <span class="token number">1</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p>If your <code>serializeJson()</code> implementation <em>always</em> returns this json, then it is deterministic:</p>
<pre class="language-json"><code class="language-json"><span class="token punctuation">{</span> <span class="token property">"A"</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token property">"B"</span><span class="token operator">:</span> <span class="token number">1</span> <span class="token punctuation">}</span></code></pre>
<p>Likewise, it is also deterministic if if <em>always</em> returns this json:</p>
<pre class="language-json"><code class="language-json"><span class="token punctuation">{</span> <span class="token property">"B"</span><span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token property">"A"</span><span class="token operator">:</span> <span class="token number">1</span> <span class="token punctuation">}</span></code></pre>
<p>Determinism isn't about the value created, it's about the consistency of that value. If the output depends on whether the struct was created using implicit object notation vs other methods (structNew, deserializeJson, etc), then it is not deterministic. A non-deterministic implementation might return the first version in some situations, and the second in others.</p>
<p>But why is this bad?</p>
<h2 id="why-do-i-care-if-the-generated-json-is-deterministic%3F" tabindex="-1">Why do I care if the generated JSON is deterministic?</h2>
<p>JavaScript objects are not ordered. From <a href="https://www.json.org/">json.org</a>: "An object is an unordered set of name/value pairs." Thus, a nondeterministic implementation of <code>serializeJson()</code> is <em>fine</em> if your goal is to create json useful to a JavaScript system or anything else that would consume that data back into its own primitives.</p>
<p>In my case, I'm using JSON as a waypoint in a chain of operations to take any hypothetical struct and <em>deterministically</em> convert it into a number between 0 and 1. Basically, I'm trying to implement this:</p>
<pre class="language-js"><code class="language-js"><span class="token function">checksum</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">foo</span><span class="token operator">:</span> <span class="token string">'bar'</span><span class="token punctuation">,</span> <span class="token literal-property property">baz</span><span class="token operator">:</span> <span class="token string">'yo'</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//returns numeric between 0-1</span></code></pre>
<p>My implementation <em>must</em> be deterministic in order to be useful. It's not all that dissimilar from calculating an MD5 hash of something, except my output is numeric and MD5 is alpha-numeric, and MD5 only accepts string inputs.</p>
<p>In fact, that is my algorithm: Convert the struct to a string, MD5 hash that string, and remove the alpha-characters, leaving only digits behind. How do you turn a given random struct to a string in CFML? There are a few ways: JSON, WDDX, or XML all immediately spring to mind. I happen to use JSON daily so it's my go-to in a situation like this.</p>
<p>MD5 is deterministic. This is one of the reasons it has been used for so long for password hashing. It wouldn't be useful in a password hashing algorithm if it weren't deterministic, because logins using the correct password would fail at random.</p>
<p>But my function <em>must</em> be deterministic, and <code>serializeJson()</code> is not; at least not on all CFML engines, and the thing I'm building will be open source and could potentially run on all modern CFML engines.</p>
<p>This is where Adam Cameron's brilliance comes in.</p>
<h2 id="how-do-you-add-determinism-where-none-exists%3F" tabindex="-1">How do you add determinism where none exists?</h2>
<p>And here we are. We've reached the crux of this article. This is so <a href="https://www.gamersdecide.com/articles/dnd-alignments-explained">"chaotic good"</a> and I absolutely love it.</p>
<p>Adam's idea was to <strong>sort the characters in the JSON output string.</strong> That's it. It's that simple.</p>
<p>Since the resulting JSON output characters (just not the order in which they appear) is deterministic, sorting the string makes it deterministic. And since I don't need the ability to deserialize back to primitives, it's totally fine.</p>
<p>Going back to the original two examples of nondeterministic JSON, after sorting they both look like this:</p>
<pre><code>,::""""{}11AB
</code></pre>
<p>Adam, you are truly an evil genius.</p>
<p>I'm looking forward to sharing more about this open source project in the near future, but thanks for letting me tell that story in the meantime.</p>
Sweat the Small Stuff2021-05-12T00:00:00Zhttps://adamtuttle.codes/blog/2021/sweat-the-small-stuff/<p>For years now we've been lecturing each other that the arguments over tabs vs. spaces and other such trivial details are meaningless bikeshedding and that we should put that behind us and get to work. In some ways I agree. In some I disagree.</p>
<p><img src="https://adamtuttle.codes/img/2021/tabs-rule.jpg" alt="tabs rule, spaces drool. change my mind." /></p>
<p>Having <a href="https://workingcode.dev/episodes/022-book-club-1-clean-code-by-uncle-bob-martin-pt1/">recently read Clean Code by Robert Martin</a>, I find myself more put off by trivially-unclean code in a code review than I was previously. Not that I am upset with my teammates for imperfect code—far from it!</p>
<p>But.</p>
<p>Imagine the codebase <em>is perfect</em>. I know, I know. Suspend your disbelief for a moment.</p>
<p>Now... you're doing a code review and the file is consistently indented with tabs but the changed line is indented with spaces. Functionally, the change is flawless. Do you reject the pull request and ask the developer to change it? <em>It's just one line!</em></p>
<p>You could approve the PR and merge it with the intention of coming back to clean it up "some day" but as we all know, "some day" is the same thing as "never."</p>
<p>Now multiply that by every pull request. Over 10 years. 20. Not just tabs-vs-spaces indentation, but other little inconsistencies, too. HTML closing tags that don't get indented to the same level as their opening tag. Lazy variable names. Outdated comments. And so on. Now remember that the code wasn't perfect to begin with.</p>
<p>This project is now a mess and <strong>harder to work on</strong>.</p>
<p>I'm not going to go through every code review with a fine-tooth comb and make sure everybody's opening and closing HTML tags are indented to matching levels, but if I notice a problem then I'm going to reject it.</p>
<p>For some languages, things like linters and code formatters can help automate this work for you. <strong>Use them!</strong> And for the love of everything holy... Turn on whitespace characters in your editor so that you can spot the inconsistencies before you submit your code review!</p>
<p>Here's how you find the setting in VS Code:</p>
<p><img src="https://adamtuttle.codes/img/2021/how-to-find-whitespace-chars.png" alt="How to turn on whitespace characters in VS Code: Search for "render whitespace"" /></p>
<p>And here's what your editor could look like with the setting enabled:</p>
<p><img src="https://adamtuttle.codes/img/2021/having-whitespace-characters-enabled.png" alt="What whitespace characters look like in VS Code" /></p>
<p>Sweating the small stuff today is an investment in reducing future stress for yourself, your team, and the countless developers that will inherit your code after you're long gone.</p>
<p>Sweat the small stuff.</p>
What I Want From My Next Tech Stack2021-05-11T00:00:00Zhttps://adamtuttle.codes/blog/2021/what-i-want-from-my-next-tech-stack/<p>I've been working on the web full time since 2005, part time since 2000, and for fun since 1997, and the vast majority of that was writing CFML ("ColdFusion"). A handful of years ago I decided I was done investing my energy into CFML and that I wanted to move on to something else. Whether or not I was a "big fish," CFML is undeniably a small pond, and I felt a calling to a larger pond.</p>
<p>The most logical next place for me seems to be full-stack JavaScript. It's been 8+ years that I've been comfortable using Node.js, but not doing it as my mainstay. I often have my fingers in AWS Lambda (always Node.js); I've written several tools, microservices, and dashboards using vanilla React and a few with Next.js; and I've dabbled with Gatsby and Eleventy.</p>
<p>And yet on average I <em>still</em> write <em>at least</em> an order of magnitude more CFML than JavaScript on any given day. But now more than ever I can see that the sun is setting on CFML in my career. <a href="https://www.alumniq.com/">My team</a> is growing, we all agree that CFML's days in our stack are numbered, and we are taking steps to migrate away from it. We have been taking these steps for almost a year now. Migrating big apps takes a <em>long time</em>.</p>
<h2 id="what-does-the-future-hold%3F" tabindex="-1">What does the future hold?</h2>
<p>As we start to look toward the future I can't help but feel that there are no good (public) examples of how to organize a <em><strong>truly large</strong></em> React.js application. <a href="https://egghead.io/q/javascript?q=organize">There is zero content on this subject on egghead.io</a>. I am intrigued by the idea of <a href="https://remix.run/">Remix</a>, but it's too young and I'm not willing to bet the farm on it yet. The most obvious choice seems to be Next.js.</p>
<p>I have a bunch of experience writing Next apps, and I certainly find a lot to like about them, but something I've never quite figured out is how to organize them for large-scale applications. Thinking about that recently, I decided it might be helpful to write down my goals for our tech stack after we leave CFML behind. So in no particular order, here's what I'm looking for:</p>
<ul>
<li>A healthy testing ecosystem: lots of tools to choose from, tests (can) run fast</li>
<li>File-system based routing, which I think helps keep the code organized</li>
<li>Turn-key bundling and code-splitting, and other similar optimizations (don't make me configure webpack, but let me tweak it if I want to)</li>
<li>Good integration options for GraphQL</li>
<li>Fast application startup time</li>
</ul>
<p>There are other things that we'll get from other aspects of the stack, like: Automated zero-downtime deploys with support for fast rollbacks, which we'll get from ECS Fargate, but I'm scoping this article to the programming language/platform/framework.</p>
<p>That's all I can think of for the moment, and I think that most of those boxes are easily checked by JavaScript and Next.js, which is why I said it seemed like an obvious choice. While it does have file-system based routing, one issue that I still struggle with in Next applications is organizing files.</p>
<h2 id="organizing-large-next.js-apps" tabindex="-1">Organizing large Next.js apps</h2>
<p>Next apps have a <code>pages/</code> directory where each route ("page") is defined.</p>
<pre class="language-js"><code class="language-js">pages<span class="token operator">/</span><br /> index<span class="token punctuation">.</span>js<br /> login<span class="token punctuation">.</span>js<br /> dashboard<span class="token punctuation">.</span>js<br /> users<span class="token operator">/</span><br /> index<span class="token punctuation">.</span>js<br /> edit<span class="token punctuation">.</span>js<br /> reports<span class="token operator">/</span><br /> a<span class="token punctuation">.</span>js<br /> b<span class="token punctuation">.</span>js<br /> c<span class="token punctuation">.</span>js<br /> <span class="token operator">...</span><br /> marketing<span class="token operator">/</span><br /> index<span class="token punctuation">.</span>js<br /> lists<span class="token operator">/</span><br /> index<span class="token punctuation">.</span>js<br /> search<span class="token punctuation">.</span>js<br /> edit<span class="token punctuation">.</span>js<br /> messages<span class="token operator">/</span><br /> index<span class="token punctuation">.</span>js<br /> calendar<span class="token punctuation">.</span>js<br /> edit<span class="token punctuation">.</span>js<br /> activity<span class="token punctuation">.</span>js</code></pre>
<p>Each of these JS files defines a Page, but don't forget that we're talking about React.js and so each page is broken down into at least a few, and possibly hundreds of reusable and shared components. The interlocking components aspect of React is fantastic and one of the reasons that I love it so much, but where do they go?</p>
<p>One nice thing about Next.js is that they let you create aliases for import/require statements, so that you can change this:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">import</span> MessageGrid <span class="token keyword">from</span> <span class="token string">'../../../../../components/marketing/messages/grid.js'</span><span class="token punctuation">;</span></code></pre>
<p>Into something that more closely resembles this:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">import</span> MessageGrid <span class="token keyword">from</span> <span class="token string">'marketing-components/messages/grid.js'</span><span class="token punctuation">;</span></code></pre>
<p>This is undeniably an improvement, but is it good enough? I don't know.</p>
<p>If the listing above was my entire application, then maybe. But it's not. My application is a Monolith with <a href="https://workingcode.dev/episodes/005-monoliths-vs-microservices/">a few microservices refactored out</a> when it made sense to do so, and that monolith has over a thousand actions that can be executed; with somewhere around half of them having views (think 800 screens with views and another 400 form-submit handlers; that'll get you close enough).</p>
<p>Hopefully it's obvious why I'm a little uneasy about this organizational structure, now? With so much functionality wrapped up into our application, colocation of our components next to the pages that use them —when specific to that portion of the application— will greatly increase maintainability. Hunting down a component location is no fun; but let's be realistic: JavaScript tooling is good enough now that you won't have to hunt for an existing component. Option+Click (or I guess Ctrl+Click for Windows devs) on it in VSCode and it will open the file for you. I'm not sure if this is true with the Next.js aliasing thing, though. 🤔</p>
<p>The other side of this same coin: Option+Click can't help you find the right folder in which to create the new component you need for the page you're working on. Colocation would be ideal. And all of this is to say nothing of the tests. My preference would be to colocate the tests with the pages, too.</p>
<p>Is this a solved problem? Just like there's no egghead content on the subject, there is <a href="https://github.com/vercel/next.js/discussions?discussions_q=organize">a curious lack of content on the Next.js discussion forum, too</a>. The first thing that comes to my mind is a file naming convention. For example, it's common to see <code>foo.spec.js</code> for a file that runs tests on the <code>foo.js</code> component. Perhaps there's something like <code>gridview.comp.js</code> that tells Next that the file is not a Page, even though it lives nested somewhere inside the <code>pages/</code> directory. There's a lot of really smart people at Vercel and they seem to work at a relentless pace, so I won't be surprised if/when they come up with something better than that.</p>
<h2 id="looking-at-it-from-another-angle" tabindex="-1">Looking at it from another angle</h2>
<p>Another question I'm asking myself is if I'm looking at this the wrong way. Yes, my current monolith envelops all of its "modules" (mail, marketing, merch, membership, and that's just the M's...) because that makes sense on its current tech stack, but in a Next.js future, does it make more sense to break them up into separate apps for each module and run each one individually?</p>
<p>I can see some benefits: Being able to work on and deploy them individually reduces opportunities for developers to get in each other's way, merge conflicts, and the like. And on the rare occasion that a catastrophic bug crashes the app, it's only the one module. But I can also see some drawbacks: Each module would need to have its own ECS service, and our total minimum number of running containers, even with absolutely zero traffic, would be one for each module; versus the monolith that could theoretically scale down to a single instance.</p>
<h2 id="this-is-only-the-beginning" tabindex="-1">This is only the beginning</h2>
<p>At any rate, this is a journey. I intend to find out the answers to these questions and I'll be sure to share what I find. If you're interested in big JavaScript apps too, maybe stick around.</p>
Dead Code2021-04-21T00:00:00Zhttps://adamtuttle.codes/blog/2021/dead-code/<p>If you didn't already know, I have a podcast (<a href="https://workingcode.dev/">Working Code</a>), and for an upcoming episode we're all reading <a href="https://amzn.to/2RLc8tb">Clean Code: A Handbook of Agile Software Craftsmanship</a> by Robert ("Uncle Bob") Martin. That's important context for what follows, because it is having a profound effect on how I am thinking about my output right now.</p>
<p>I won't be sharing my thoughts on the book here (for that, listen to episode 22 which will publish on May 12th), but one of the suggestions (heh) of the book is that, to summarize, <strong>commented blocks of code are evil</strong>; and it is that concept that I want to explore today.</p>
<p>In general, I would tend to agree; and I've given the same advice plenty of times over the years. But I want to explore some commented code from one of my modules that continues to survive today and, I think, for good reason.</p>
<h2 id="a-problem-with-many-possible-solutions" tabindex="-1">A Problem With Many Possible Solutions</h2>
<p>Suppose that you need to compare some arrays and find only the items that appear in all of the input arrays. In the venn diagram, it's the intersection of all circles. Each item is a string of up to 100 characters. Now suppose that each array holds somewhere between 0 and 500,000 rows; and that you have somewhere between 2 and 100 of these arrays. The goal is to perform this calculation as fast as possible. That was my assignment.</p>
<p>Actually, I had the luxury that originally we didn't anticipate the number of arrays being so large, nor the number of items in each array; so my original implementation was naive because the requirements were naive -- but it worked.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">private</span> array <span class="token keyword">function</span> <span class="token function">arrayIntersectionByCounting</span><span class="token punctuation">(</span> <span class="token parameter">required array inputArrays</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">var</span> hashtable <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span><br /> <span class="token keyword">var</span> numberOfInputs <span class="token operator">=</span> <span class="token function">arrayLen</span><span class="token punctuation">(</span> arguments<span class="token punctuation">.</span>inputArrays <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">var</span> intersectionResults <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token comment">//count instances of each array value</span><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> currentArray <span class="token keyword">in</span> arguments<span class="token punctuation">.</span>inputArrays<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//if any array is empty, the result will be an empty array,</span><br /> <span class="token comment">//so just short-circuit the whole process</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">arrayLen</span><span class="token punctuation">(</span>currentArray<span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> i <span class="token keyword">in</span> currentArray<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span><span class="token function">structKeyExists</span><span class="token punctuation">(</span>hashtable<span class="token punctuation">,</span>i<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> hashtable<span class="token punctuation">[</span>i<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token keyword">else</span><span class="token punctuation">{</span><br /> hashtable<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token operator">++</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token comment">//make a new array of values that appear the minimum number of times</span><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> k <span class="token keyword">in</span> hashtable<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>hashtable<span class="token punctuation">[</span>k<span class="token punctuation">]</span> <span class="token operator">==</span> numberOfInputs<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token function">arrayAppend</span><span class="token punctuation">(</span>intersectionResults<span class="token punctuation">,</span> k<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">return</span> intersectionResults<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>This implementation uses the tools at our disposal: the language native data structures and simple but efficient looping. It outputs the correct results, but once the system started to grow and the input arrays became longer and more numerous, the runtime for calculations grew to unacceptable lengths. So additional approaches were tried.</p>
<p>I won't go into the details of each, but suffice it to say, we racked our brains and tried really hard to squeeze every ounce of performance out of this stone.</p>
<ul>
<li><strong>arrayIntersectionByCFMLHashtable</strong>: sorts the input arrays by length and then compares the shortest with the next shortest, returning only the items that exist in both; then repeat with the result of the previous iteration and the next-shortest array until complete.</li>
<li><strong>arrayIntersectionByJavaHashset</strong>: Maybe dropping down to the Java layer can help us?</li>
<li><strong>arrayIntersectionByJavaRetainAll</strong>: What about Java's <code>array.retainAll()</code> method?</li>
<li><strong>arrayIntersectionByReference</strong>: Passing arrays in CFML is done by value, which uses more memory and one would assume, extra time to make the copies. In an attempt to minimize memory usage and speed things up, this implementation wrapped every array in a structure (for the non-CFML folks: a Hashmap or an Object) because that would be passed by reference instead. Similar to arrayIntersectionByCFMLHashtable, but different. The algorithm it used was to find the shortest input array and then search for each of its items in all of the other arrays. Whenever an item was not found in an array, immediately move on to the next item. This one should be pretty performant at least in terms of Big-O notation, but still wasn't good enough.</li>
<li><strong>arrayIntersectionBySQL</strong>: Pump all of the data into temp tables in the database and let SQL do the heavy lifting via an inner-join.</li>
<li><strong>arrayIntersectionByQueryOfQueries</strong>: a CFML feature for in-memory data manipulation using its native Query data type. Kind of a long shot.</li>
</ul>
<p>Each of these had their merits, but ultimately I found something that beat them all. It's been years so I don't recall the exact numbers of how one method performed in comparison to the others, but ultimately what we ended up going with was <code>arrayIntersectionByRedis</code>. This solution was similar in concept to the SQL attempt, but instead of letting MySQL do the heavy lifting, we let Redis do it. Redis has a data structure called a <strong>set</strong> and the command <a href="https://redis.io/commands/sinter">sinter</a> performs an intersection of N sets. Through some form of arcane wizardry, it's surprisingly performant.</p>
<h2 id="gravestones" tabindex="-1">Gravestones</h2>
<p>I'm not providing the implementation for <code>arrayIntersectionByRedis</code> here because (1) that's not what I want to talk about and (2) it's a little bit on the longer side and with some basic research on Redis Sets I'm sure you could figure it out. No, what I'm interested in right now is the journey.</p>
<p>No doubt, if each of those failed solutions were deleted when a better idea came along, because the code was dead -- as it would seem that Uncle Bob is advocating -- then in somewhere between 6 months to 60 years, someone (probably me) would come along and see this weird implementation and think to themselves, "that's odd. I bet it would be faster to do X..." and they would be completely unaware that X (and Y and Z and P and D and Q) was already tried.</p>
<p>By leaving behind not only the names of things that were tried in this particularly pernicious problem, but their implementation details too, all with a note at the top of the comment block that explains why the code is commented instead of deleted, that developer (future me) can be reassured that they have in fact not had a new idea and that, should they want to try something, they should be prepared to carefully measure the performance differences.</p>
<p>Those commented out implementations are <strong>gravestones</strong>. They are nearly as useful as the living, breathing code. They warn drive-by developers from thinking they can clever up a better solution without careful, measured dilligence.</p>
<p>Indeed, I thought I had found just such a case at one point. I found a way to do some stuff in the database that seemed, for <em>days of actual testing and measuring</em> to be more performant. I made the change in a branch, and made a PR with a mile-long explanation including screen shots and gifs and a very careful explanation of the problem and <em>why</em> my new solution was better. And then, right as I was building to the conclusion of the PR description, I put my code onto a QA server and ran it there for more production-like performance numbers and, wouldn't you know it, the new solution was rarely any better at all than the one that used Redis, and most of the time equal or slower.</p>
<h2 id="on-rules" tabindex="-1">On Rules</h2>
<p>As will become evident in the upcoming Book Club episode of <a href="https://workingcode.dev/">Working Code</a> (Episode 22 on May 12th!), I think that it's super valuable to know what the rules are, but that one of the values of knowing and understanding the rules is so that you know when it makes sense to break them.</p>
<p>You could make the case that instead of leaving the code behind in a comment block, it should be moved to a wiki or an issue/ticket, and that document should be linked in a comment instead. I wouldn't necessarily disagree, but at the time my team didn't have a great ticketing system (oh how quickly we forget about the time before GitHub!) and I have my doubts about whether a single line comment would grab attention like a giant block of commented functions. It's so rare in our codebase that it demands attention.</p>
My Ongoing Love Affair With GNU Make2021-03-19T00:00:00Zhttps://adamtuttle.codes/blog/2021/my-ongoing-love-affair-with-gnu-make/<p>I've written previously about <a href="https://adamtuttle.codes/blog/2020/how-I-use-make/">my love for Make</a>. The longer I use it, the more complex my desires become, and the more I learn about the awesome features (and hacks) available.</p>
<p>Since that previous article, my team has been using a heck of a lot of Makefiles to work on automations, and combining them with <a href="https://github.com/features/actions">GitHub Actions</a> for things like testing and deployment automations. We developed some patterns, and as developers do, we started thinking about making that code reusable.</p>
<p>Rather than having to update 23 makefiles every time we come up with a process improvement, what if we could update 1 place and they would automatically (or at least <em>easily</em>) inherit it?! I found <a href="https://mattandre.ws/2016/05/makefile-inheritance/">an article covering exactly that</a>. Basically, you can add <code>include somefile</code> at the top of your makefile and it will be imported and treated as if it were already there.</p>
<p>Given the hammers that were already in our hands, our first approach was to create a dedicated GitHub Repo with that "base" makefile, and treat it as an npm module. We already use npm and node modules extensively, and given these two choices for import lines:</p>
<pre class="language-bash"><code class="language-bash">include base.Makefile</code></pre>
<p>or</p>
<pre class="language-bash"><code class="language-bash">include node_modules/alumniq_devops/base.Makefile</code></pre>
<p><strong>... Who cares?</strong></p>
<p>So we started down that path. Unfortunately, that's also where we hit a snag. Not <em>every</em> one of our projects uses node.js and npm. And to have to add a package.json and require an <code>npm ci</code> step so that we can then run <code>make</code> does have a bit of a bad smell to it. And in the worst case an <code>npm ci</code> was required to get the base makefile, so that make could run, which would then need to run <em>another</em> <code>npm ci</code>. There's a lot to like about node.js, but the speed of npm isn't really one of them.</p>
<p>Our next thought was to try and access the <code>base.Makefile</code> directly from <code>raw.githubusercontent.com</code>, hoping that we could use http-basic auth with an oauth token, <a href="https://docs.npmjs.com/cli/v6/configuring-npm/package-json#git-urls-as-dependencies">like you can for private git-url based npm modules</a>, to download the file; skipping the npm install step (and not even requiring node or npm, just curl).</p>
<p>Sure, we could publish the base makefile in a public repo, but we keep most of our code private, and we would prefer that this base Makefile be no different. It contains some private things like our ECR repository URL. It wouldn't be catastrophic if it were shared, but it's not something we want to volunteer, either. Keeping the file contents private is a pretty high priority.</p>
<p>My next thought was to try a gist, because those can be private but you only need the URL to access it, no credentials. In addition, on a complete lark, I hoped that Make would see that the file was missing and check to see if there is a target that could create it. Amazingly, this works!</p>
<pre class="language-bash"><code class="language-bash">include base.Makefile<br /><br />base.Makefile:<br /> @curl https://gist.githubusercontent.com/atuttle/507151305ad440902c64df2f24145c90/raw/4a3b8fe156c64faa3a444f7d79a290fd9b28d707/ <span class="token parameter variable">-o</span> base.Makefile <span class="token operator"><span class="token file-descriptor important">2</span>></span>/dev/null</code></pre>
<p>But this approach has its own problem: when the gist gets updated, that url doesn't go to the new revision, it's pointing directly to the old revision. The whole point of this exercise was to be able to update the base makefile and have all of the projects using it automatically get the update.</p>
<p>The solution I came up with isn't a full automatic-update, but it's darn close. Having recently made my own url shortener, I knew that I could create a permanent URL that would redirect to the current gist URL, and I could update it as I see fit. Of course that's not how URL shorteners are supposed to work, but nobody using this URL would <em>prefer</em> to get the old URL, so it's fine.</p>
<pre class="language-bash"><code class="language-bash">include base.Makefile<br /><br />base.Makefile:<br /> @curl https://tutt.xyz/def-not-a-fake-url <span class="token parameter variable">-o</span> base.Makefile <span class="token operator"><span class="token file-descriptor important">2</span>></span>/dev/null</code></pre>
<p>I tried to add the <code>curl</code> command to the root of the file, but you can't have commands there (other than include?), so that doesn't work.</p>
<p>So you might be asking yourself what about updates? One of the targets in <code>base.Makefile</code> is:</p>
<pre class="language-bash"><code class="language-bash">update:<br /> @curl https://tutt.xyz/def-not-a-fake-url <span class="token parameter variable">-o</span> base.Makefile <span class="token operator"><span class="token file-descriptor important">2</span>></span>/dev/null</code></pre>
<h2 id="holy-inheritance%2C-batman!" tabindex="-1">Holy Inheritance, Batman!</h2>
<p>The <em>other</em> awesome thing I learned about Makefiles this week was an <em>even better</em> approach to overriding targets. Much of the initial work on figuring out these details was done by my teammate Adam Crump, so, thanks Adam!</p>
<p>If you start with a <code>base.Makefile</code>:</p>
<pre class="language-bash"><code class="language-bash">foo:<br /> @echo <span class="token string">"parent-foo"</span></code></pre>
<p>And this is your <code>Makefile</code>:</p>
<pre class="language-bash"><code class="language-bash">include base.Makefile<br /><br />foo:<br /> @echo <span class="token string">"child-foo"</span></code></pre>
<p>If you run <code>$ make foo</code> then you can see here that it works, but you get some unsightly warnings:</p>
<pre class="language-bash"><code class="language-bash">Makefile:4: warning: overriding commands <span class="token keyword">for</span> target <span class="token variable"><span class="token variable">`</span>foo'<br />base.Makefile:2: warning: ignoring old commands <span class="token keyword">for</span> target <span class="token variable">`</span></span>foo'<br />child-foo</code></pre>
<p>It turns out, there's a <a href="https://stackoverflow.com/a/49804748/751">neat hack</a> to make those warnings go away, and it also allows you to call the version from the base makefile from the child, if you happened to want to. For example, you might want to run an <code>npm ci</code> before you call the target.</p>
<p>At the bottom of your <code>base.Makefile</code>, add this:</p>
<pre class="language-bash"><code class="language-bash">%: %-super<br /> @ <span class="token boolean">true</span></code></pre>
<p>Basically this is a wildcard match. If no other targets match the requested target, then the wildcard target runs, and it depends on the target of the same name with the <code>-super</code> suffix.</p>
<p>So... then add the suffix: <code>-super</code> to all of the targets in your <code>base.Makefile</code>:</p>
<pre class="language-bash"><code class="language-bash">foo-super:<br /> @echo <span class="token string">"parent-foo"</span><br /><br />bar-super:<br /> @echo <span class="token string">"parent-bar"</span><br /><br />baz-super:<br /> @echo <span class="token string">"parent-baz"</span><br /><br />%: %-super<br /> @ <span class="token boolean">true</span></code></pre>
<p>And if you want to override one, but then also run the original implementation, you can call it directly with the <code>make target-super</code> full name. So in this example, <code>make bar</code> will also call <code>make bar-super</code>:</p>
<pre class="language-bash"><code class="language-bash">include base.Makefile<br /><br />foo:<br /> @echo <span class="token string">"child-foo"</span><br /><br />bar:<br /> @echo <span class="token string">"child-bar"</span><br /> @make bar-super</code></pre>
<p>And now, you can run all three of these targets (<code>foo</code>, <code>bar</code>, and <code>baz</code>) with no warnings. Note that <code>baz</code> is NOT overridden, and only exists with the <code>-super</code> prefix, but I can still run it with the command <code>make baz</code></p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">make</span> foo<br />child-foo<br /><br />$ <span class="token function">make</span> bar<br />child-bar<br />parent-bar<br /><br />$ <span class="token function">make</span> baz<br />parent-baz</code></pre>
<p>I'm super excited about the possibilities this opens up for my team.</p>
Faster Error Navigation From Lucee Stack Traces2021-01-29T00:00:00Zhttps://adamtuttle.codes/blog/2021/faster-error-navigation-lucee-stack-traces/<p>I wrote a helpful and tiny automation that I want to share with you. If you use the <a href="https://www.lucee.org/">Lucee</a> CFML application server, and a code editor that supports the same advanced "go to file" syntax as <a href="https://code.visualstudio.com/">VSCode</a>, it might make your life a little bit easier.</p>
<p>The Lucee part is kind of obvious; either you're using it or you're not.</p>
<p>The VSCode-style "go to file syntax" is less obvious. Specifically, what I'm referring to is the go-to-file command (cmd-p on Mac, ctrl-p on Windows) where you can do a fuzzy search of the file name, and then once you've located the right file, you can append either <code>@symbol</code> or <code>:line-number</code> to the file name to jump directly to that symbol (e.g. function name, variable, etc) or line number in the file. Credit where it's due, the first time I remember seeing this done was in Sublime Text.</p>
<p><img src="https://adamtuttle.codes/img/2021/vscode-go-to-file.png" alt="screen shot of VSCode go-to-file dialog while searching for "appcfc@onError"" /></p>
<p>Do you see where this is going yet?</p>
<p><img src="https://adamtuttle.codes/img/2021/cfml-dump.png" alt="screen shot of a portion of a Lucee stack trace dump" /></p>
<p>We have an editor capable of doing advanced searches for path and file name and taking you directly to a specific line number; and we have an error stack trace that includes the file path and line number in almost exactly the format we would use to load it in the editor...</p>
<p>For I-don't-even-want-to-imagine-how-many-years-of-my-life, now (ugh!) I've been getting that file name via copy and paste like a chump, or worse, typing it manually, with my hands, like a savage.</p>
<p>I would rather click on the file name in the stack trace and have it open directly to that location in VSCode. If I could do that, that would be swell. I might even work on that. But for now, here's what I've already done, because it was <em>easy</em>:</p>
<p>Add this to a page that includes a Lucee stack trace dump, and clicking on the box with the file path will copy it to your clipboard. I added it to my team's error page that we see in local development and QA environments when there's an error.</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>style</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>text/css<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token style"><span class="token language-css"><br /> <span class="token selector">.luceeN1</span> <span class="token punctuation">{</span><br /> <span class="token property">cursor</span><span class="token punctuation">:</span> pointer<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>style</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>script</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>text/javascript<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script"><span class="token language-javascript"><br /> <span class="token function">$</span><span class="token punctuation">(</span>document<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">on</span><span class="token punctuation">(</span><span class="token string">'click'</span><span class="token punctuation">,</span> <span class="token string">'.luceeN1'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token parameter">e</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">try</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> initialText <span class="token operator">=</span> <span class="token function">$</span><span class="token punctuation">(</span>e<span class="token punctuation">.</span>target<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">html</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">const</span> path <span class="token operator">=</span> initialText<span class="token punctuation">.</span><span class="token function">split</span><span class="token punctuation">(</span><span class="token string">'('</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">.</span><span class="token function">split</span><span class="token punctuation">(</span><span class="token string">')'</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>path<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">===</span> <span class="token string">'/'</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token comment">//everything after the first character</span><br /> navigator<span class="token punctuation">.</span>clipboard<span class="token punctuation">.</span><span class="token function">writeText</span><span class="token punctuation">(</span>path<span class="token punctuation">.</span><span class="token function">slice</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token comment">//the whole thing</span><br /> navigator<span class="token punctuation">.</span>clipboard<span class="token punctuation">.</span><span class="token function">writeText</span><span class="token punctuation">(</span>path<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">{</span><br /> navigator<span class="token punctuation">.</span>clipboard<span class="token punctuation">.</span><span class="token function">writeText</span><span class="token punctuation">(</span><span class="token function">$</span><span class="token punctuation">(</span>e<span class="token punctuation">.</span>target<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">html</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /></span></span><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>script</span><span class="token punctuation">></span></span></code></pre>
<p>It's kind of naïve in its implementation, granted. You have to be using a modern enough browser for <a href="https://caniuse.com/?search=navigator.clipboard">navigator.clipboard support</a>, and it applies all of this logic to every light-orange box when really it only does anything useful for the few of them that we're narrowly concerned with... But it works!</p>
<p>It assumes that jQuery is available on the page. In my case, it always will be, so why not use it? Then, on click, it grabs the contents of the first thing it can find between parenthesis. Actually, if there is a second open-parenthesis <code>(</code> before the first close-parenthesis <code>)</code> then it will grab just what's between those first two open-parenthesis. Like I said... it's naïve!</p>
<p>Whatever it finds, it copies that to your clipboard, making it easy to paste into VSCode and open exactly that file to exactly the line in question.</p>
<p>One other little thing. In my screen shot you can see that the file path begins with a slash: <code>/admin/Application.cfc:145</code>. If that admin folder is in the root of your project, VSCode won't find the file because of the slash, and it hurts nothing to remove the leading slash, so I do. If you're on Windows I imagine the paths will be different and you may need to reverse the slashes, and some other things, but hopefully this can get you started.</p>
Sign Your Git Commits2020-11-05T00:00:00Zhttps://adamtuttle.codes/blog/2020/sign-your-git-commits/<p><img src="https://adamtuttle.codes/img/2020/david-nitschke-pegxjW_1YOU-unsplash.jpg" alt="Wax seal" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@david_nitschke_95?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">David Nitschke</a> on <a href="https://unsplash.com/s/photos/signature?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>Until today I was complacent about signing my git commits. Who cares if they're signed?</p>
<p>Well, it matters.</p>
<p>Git will allow you to associate any name and email address with a commit, without any oversight to ensure you have that name and control that email address. And that's fine, because git is not an email platform. Where this becomes problematic is spoofing -- and that's the part I didn't get until today. I can create commits and attribute them to Bill Gates or Steve Ballmer or <em>you</em> if I know your name and email address. And whose email address is truly private any more?</p>
<p>Recently, via DMCA takedown request, the popular <code>youtube-dl</code> client for downloading youtube videos was at least temporarily removed from GitHub. What's interesting about this is what happened next. GitHub uses <a href="https://github.com/github/dmca">a repo named DMCA</a> to manage DMCA requests and notify its users when they're affected. In retalliation for the RIAA causing a beloved archival tool, someone made a pull request to the DMCA repo to add the full <code>youtube-dl</code> source to it. Of course it won't be accepted, but if you have <a href="https://github.com/github/dmca/tree/416da574ec0df3388f652e44f7fe71b1e3a4701f">the right link</a> then it looks like it was merged and the repo contents were replaced, unless you notice the URL.</p>
<p>Kind of a big middle finger to the whole thing. It's kind of immature, but whatever. I get it. I laughed it off and went about my business.</p>
<p>Then today I saw that there was supposedly some sort of <a href="https://www.reddit.com/r/programming/comments/joa39m/github_source_code_leaked_online/">"leak" of the GitHub source code</a> by a similar means. It was <a href="https://www.reddit.com/r/DataHoarder/comments/jnzxmd/someone_pushed_github_source_code_to_their_dmca/gb5unub/">apparently</a> available on <a href="http://archive.org/">archive.org</a>'s wayback machine for a while, but it's been removed from there and also from the similar link that would show the code (as the <code>youtube-dl</code> link still does)...</p>
<p>This screen shot of the commit proposed for merge into DMCA is what opened my eyes:</p>
<p><img src="https://adamtuttle.codes/img/2020/github-spoof.png" alt="A spoofed commit from Nat Friedman, CEO of GitHub, supposedly leaking the GitHub source into the DMCA repo" /></p>
<p>Without a signature, there's no way to know if this was really committed by Nat Friedman. Of course it wasn't, but there's no good way to prove it. Git has a way to let you provide that proof: Signing your commits. The signature verifies that it was you, the owner of the key, that made the commit.</p>
<h2 id="how-to-sign-your-commits" tabindex="-1">How to sign your commits</h2>
<blockquote>
<p><strong>November 2022 update:</strong> I have switched from GPG signing to using SSH signatures and 1Password to manage them. 1Password provides a better experience, including using my mac's Touch-Id to authenticate the key for commits, and they have <a href="https://blog.1password.com/git-commit-signing/">a great guide to setting it up</a>.</p>
</blockquote>
<p>There are plenty of guides out there, including <a href="https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/managing-commit-signature-verification">the one I used</a> provided by GitHub, so I'm going to make this short and sweet. If this isn't enough for you, or if you're not on a Mac, I suggest you start by looking at <a href="https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/managing-commit-signature-verification">the GitHub guide</a>.</p>
<p>Do you have <code>gpg</code> installed?</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">which</span> gpg</code></pre>
<p>If not, install it:</p>
<pre class="language-bash"><code class="language-bash">$ brew <span class="token function">install</span> gpg</code></pre>
<p>Check to make sure you don't already have a key:</p>
<pre><code>$ gpg --list-secret-keys --keyid-format LONG
</code></pre>
<p>If you need to generate a key:</p>
<pre class="language-bash"><code class="language-bash">gpg --full-generate-key</code></pre>
<p>You'll be prompted for a key type. I went with: <code>RSA and RSA (default)</code>.</p>
<p>For size, choose at least 4096.</p>
<p>Specify a length of time for the key to be valid.</p>
<p>When prompted for your email address, be sure to use one that's associated with your GitHub profile and verified with GitHub.</p>
<p>After your key is created, you'll need to get its ID for the next step. In this example, the ID value you need to get is <code>3AA5C34371567BD2</code></p>
<pre class="language-bash"><code class="language-bash">$ gpg --list-secret-keys --keyid-format LONG<br />/Users/hubot/.gnupg/secring.gpg<br />------------------------------------<br />sec 4096R/3AA5C34371567BD2 <span class="token number">2016</span>-03-10 <span class="token punctuation">[</span>expires: <span class="token number">2017</span>-03-10<span class="token punctuation">]</span><br />uid Hubot<br />ssb 4096R/42B317FD4BA89E7A <span class="token number">2016</span>-03-10</code></pre>
<p>Display the ascii-armor formatted version of your new key:</p>
<pre class="language-bash"><code class="language-bash">$ gpg <span class="token parameter variable">--armor</span> <span class="token parameter variable">--export</span> 3AA5C34371567BD2</code></pre>
<p>Copy the output, beginning with <code>-----BEGIN PGP PUBLIC KEY BLOCK-----</code> and ending with<br /> <code>-----END PGP PUBLIC KEY BLOCK-----</code>. Include those markers in what you copy.</p>
<p>In your GitHub profile, go to Settings > SSH and GPG Keys. Click the "New GPG key" button. Paste in the content you just copied, and save it. GitHub will now recognize any commits signed with that key as having been signed by you.</p>
<p>Next, we need to tell git to sign all of your commits. I don't see any reason why you wouldn't want to sign all of your commits to every repo going forward, so here's how you set the setting globablly. Note that we're again referencing the GPG key ID value you copied above:</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> config <span class="token parameter variable">--global</span> user.signingkey 3AA5C34371567BD2</code></pre>
<p>This works and if you don't mind typing in your password every time you commit something, you're done. My computer is well-protected enough that I want my password to be remembered for commits. For this situation, GitHub recommends installing <a href="https://gpgtools.org/">GPG Suite</a> which allows you to save the gpg key password into your keychain, which if I understand correctly, means that you'll only need to enter your keychain password (system login password) after a long period of inactivity/etc; and that for multiple commits a few minutes apart signing will just happen invisibly.</p>
React Hooks Cheat Sheet2020-10-23T00:00:00Zhttps://adamtuttle.codes/blog/2020/react-hooks-cheat-sheet/<p>I just finished watching <a href="https://egghead.io/playlists/react-hooks-revisited-abce">this excellent video series on Egghead</a>. I found that it really simplified a lot of the hard parts of <code>useEffect</code> and also explained a few of the other hooks that I haven't taken the time to learn yet. I was so impressed by it that I was inspired to write down an even shorter version for later reference to help reinforce the lessons, and also to share with you. If any of this feels like there's not enough detail for you, I would highly encourage you to check out <a href="https://egghead.io/playlists/react-hooks-revisited-abce">the videos on egghead</a>, where Ryan goes into more detail and gives examples for each.</p>
<h2 id="usestate" tabindex="-1">useState</h2>
<p>I'm not going to cover <code>useState</code> because that's out of scope for what I want to accomplish here, but it is a pre-requisite for understanding what's below. If that one doesn't make sense to you already, you won't understand the rest of these.</p>
<h2 id="useeffect" tabindex="-1">useEffect</h2>
<p>useEffect has three different usage modes, and is used to synchronize React with anything that React doesn't control; from the page title to making AJAX requests or dealing with device api's (like getUserMedia).</p>
<ol>
<li>No 2nd argument: Executes on every render.</li>
</ol>
<pre class="language-js"><code class="language-js"><span class="token function">useEffect</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> document<span class="token punctuation">.</span>title <span class="token operator">=</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span><span class="token keyword">new</span> <span class="token class-name">Date</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toLocaleString</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<ol start="2">
<li>Array of variables as 2nd argument: Executes any time any of those variables change.</li>
</ol>
<pre class="language-js"><code class="language-js"><span class="token function">useEffect</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token comment">/* there's a new value for someData */</span><br /><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">[</span>someData<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<ol start="3">
<li>Empty array as 2nd argument: Executes only on FIRST render, and never again until page refresh.</li>
</ol>
<pre class="language-js"><code class="language-js"><span class="token function">useEffect</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> window<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'resize'</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>In this case, I'm attaching an event listener to the window's resize event, and if I re-run that every time the component re-renders, I'll be attaching a new listener every time, effectively calling my listener potentially hundreds or thousands of times every time the event occurs.</p>
<p>If your effect creates something that needs to be cleaned up when the component is removed (like removing created event listeners), the effect should return a function that does that cleanup work, and React will run it only when the cleanup is needed. Since that's exactly what I did in the last code block above, let's fix it to be correct here:</p>
<pre class="language-js"><code class="language-js"><span class="token function">useEffect</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> window<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'resize'</span><span class="token punctuation">,</span> handleResize<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> window<span class="token punctuation">.</span><span class="token function">removeEventListener</span><span class="token punctuation">(</span><span class="token string">'resize'</span><span class="token punctuation">,</span> handleResize<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<h2 id="useref" tabindex="-1">useRef</h2>
<p><em>Not just for persisting a reference to an input element!</em></p>
<p><code>useRef</code> can be treated similarly to <code>useState</code>, except that it doesn't trigger a re-render. If your render function displays the ref value, the rendered content won't be updated when the ref is updated, but if something triggers a re-render then the latest value from the ref is available to the render function. You can use this to track data changes in a more performant way, as long as you don't need those changes to be re-rendered.</p>
<h2 id="usememo" tabindex="-1">useMemo</h2>
<p>Memoization is caching the results of a pure function, which can be helpful if that function requires intensive or long-running calculations. While implementing memoization is a simple and well understood pattern, <code>useMemo</code> combines that with the API of useEffect so that it only ever runs if the input values change.</p>
<pre class="language-js"><code class="language-js"><span class="token comment">//not only will useMemo cache the results for the current value of lotsOfData</span><br /><span class="token comment">//but it will also ONLY run on the first render or after lotsOfData changes</span><br /><span class="token keyword">const</span> <span class="token function-variable function">intenseMath</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token number">42</span><span class="token punctuation">;</span><br /><span class="token keyword">const</span> usefulData <span class="token operator">=</span> <span class="token function">useMemo</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">intenseMath</span><span class="token punctuation">(</span>lotsOfData<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">[</span>lotsOfData<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<h2 id="usecallback" tabindex="-1">useCallback</h2>
<p>Whereas <code>useMemo</code> returns a memoized value, <code>useCallback</code> returns a memoized function. When you want to be able to pass the memoized callback function around as a property, use <code>useCallback</code> instead. Via closure this helps you expose functionality without exposing internal state.</p>
<h2 id="uselayouteffect" tabindex="-1">useLayoutEffect</h2>
<p><code>useLayoutEffect</code> is different from <code>useEffect</code> in that it runs <em>before</em> the DOM paints. This allows you to make changes that will affect what your app looks like (positioning elements, for example), without causing the screen to flash with a repaint because you moved something immediately after it was painted.</p>
<h2 id="usecontext" tabindex="-1">useContext</h2>
<p>Contexts have been around for a while now as a useful way to work around prop drilling. <code>useContext</code> makes it simple to create and access contexts wherever you need them.</p>
<p>Creating and applying the provider:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">export</span> <span class="token keyword">const</span> FooContext <span class="token operator">=</span> React<span class="token punctuation">.</span><span class="token function">createContext</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">function</span> <span class="token function">FooComponent</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token punctuation">(</span><br /> <span class="token operator"><</span>FooContext<span class="token punctuation">.</span>Provider value<span class="token operator">=</span><span class="token punctuation">{</span><span class="token punctuation">{</span> data<span class="token punctuation">,</span> setSomething <span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token operator">></span><br /> <span class="token operator"><</span>Foo <span class="token operator">/</span><span class="token operator">></span><br /> <span class="token operator"><</span><span class="token operator">/</span>FooContext<span class="token punctuation">.</span>Provider<span class="token operator">></span><br /> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Using properties of the context elsewhere:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">import</span> <span class="token punctuation">{</span> FooContext <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">'./FooComponent'</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">function</span> <span class="token function">SomeFooChild</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> context <span class="token operator">=</span> <span class="token function">useContext</span><span class="token punctuation">(</span>FooContext<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> <span class="token operator"><</span>div<span class="token operator">></span><span class="token punctuation">{</span>context<span class="token punctuation">.</span>data<span class="token punctuation">}</span><span class="token operator"><</span><span class="token operator">/</span>div<span class="token operator">></span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<h2 id="usereducer" tabindex="-1">useReducer</h2>
<p>This one is like a mashup of <code>useState</code> and <a href="https://github.com/reduxjs/redux">Redux</a>. You dispatch actions that are all handled by a single (composable if you want) reducer, to update a state object containing multiple values. Generally considered a best practice not to group values that aren't related. For example, groupe the state for inputs in a form.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> <span class="token punctuation">[</span>state<span class="token punctuation">,</span> dispatch<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useReducer</span><span class="token punctuation">(</span>reducer<span class="token punctuation">,</span> initialValue<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">const</span> <span class="token function-variable function">reducer</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">state<span class="token punctuation">,</span> <span class="token punctuation">{</span> type<span class="token punctuation">,</span> value <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">switch</span> <span class="token punctuation">(</span>type<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">case</span> <span class="token string">'foo'</span><span class="token operator">:</span><br /> <span class="token keyword">return</span> <span class="token punctuation">{</span><br /> <span class="token operator">...</span>state<br /> <span class="token comment">//plus whatever changes are indicated by</span><br /> <span class="token comment">//type=foo and the associated value</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /> <span class="token keyword">default</span><span class="token operator">:</span><br /> <span class="token keyword">return</span> state<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /><span class="token comment">//to dispatch an event:</span><br /><span class="token function">dispatch</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">type</span><span class="token operator">:</span> <span class="token string">'foo'</span><span class="token punctuation">,</span> <span class="token literal-property property">value</span><span class="token operator">:</span> <span class="token number">42</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<h2 id="usedebugvalue" tabindex="-1">useDebugValue</h2>
<p>Lastly, <code>useDebugValue</code> allows you to highlight something in the <a href="https://chrome.google.com/webstore/detail/react-developer-tools/fmkadmapgofadopljbjfkapdkoienihi">React Dev Tools</a> <strong>from inside a custom hook</strong>. Think of it like <code>console.log</code> but better. When your debug value is large or complex, you can improve app performance by passing a 2nd argument to useDebugValue. The 2nd argument is a function and is used to format the value. This can be a performance benefit because the format function is only called if the value is inspected in the dev tools.</p>
<pre class="language-js"><code class="language-js"><span class="token function">useDebugValue</span><span class="token punctuation">(</span>myState<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token parameter">s</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token constant">JSON</span><span class="token punctuation">.</span><span class="token function">stringify</span><span class="token punctuation">(</span>s<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>This shows up in the devtools as a key-value pair, and the key name is the name of your custom hook.</p>
<h2 id="wrapping-up" tabindex="-1">Wrapping Up</h2>
<p>As I mentioned at the top, this was a heavily summarized version of what I learned from Ryan Harris' egghead course, <a href="https://egghead.io/playlists/react-hooks-revisited-abce">React Hooks: Revisited</a>. Thanks to Ryan for a fantastic quick video course and his simple and clear explanations!</p>
How I Use Make to Automate My Development Environment2020-09-21T00:00:00Zhttps://adamtuttle.codes/blog/2020/how-I-use-make/<p><img src="https://adamtuttle.codes/img/2020/kevin-ku-w7ZyuGYNpRQ-unsplash.jpg" alt="A blurry computer screen in the background, and eyeglasses in the foreground, through which the computer screen is in focus" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@ikukevk?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Kevin Ku</a> on <a href="https://unsplash.com/s/photos/hacker?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>I don't know if this is brilliant or boneheaded, but it works reasonably well for me, so I thought I'd share.</p>
<p>A long time ago I saw my friend (and super smart dude) <a href="https://twitter.com/neurotic">Mark Mandel</a> use Make as a way to store command aliases within a project. Maybe that sentence needs some explanation.</p>
<h2 id="aliasing-cli-commands" tabindex="-1">Aliasing CLI Commands</h2>
<p>The command line interface (CLI) can be an incredibly powerful environment, but to fully use that power you might need to string together an incredibly long command. For example, after a <code>git merge</code> you'll sometimes have merge conflicts. Here's a command that will open them all in VSCode for editing:</p>
<pre class="language-shell"><code class="language-shell"><span class="token function">git</span> status <span class="token parameter variable">-sb</span> <span class="token operator">|</span> <span class="token function">grep</span> UU <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{print $2}'</span> <span class="token operator">|</span> <span class="token function">grep</span> <span class="token parameter variable">-v</span> <span class="token string">'\.min\.'</span> <span class="token operator">|</span> <span class="token function">xargs</span> code</code></pre>
<p>This is several commands chained together:</p>
<ol>
<li><code>git status -sb</code> Gets a list of files from git that are in a modified state</li>
<li><code>grep UU</code> Excludes anything that's not conflicted -- <code>git status -sb</code> prefixes conflicted files with <code>UU</code></li>
<li><code>awk '{print $2}'</code> prints the 2nd column (space-delimited) from the line, so just the filename</li>
<li><code>grep -v '\.min\.'</code> excludes minified files (e.g. JavaScript builds), because a build fixes those.</li>
<li><code>xargs code</code> runs the <code>code</code> command for each line of input, in this case a conflicted file.</li>
</ol>
<p>But there's no way in hell I'm going to remember all of that, let alone type it all out perfectly, every time I have merge conflicts... and that's only just scratching the surface of the universe of long and complex CLI commands.</p>
<p>So I have this saved as an alias in my <code>.zshrc</code> file:</p>
<pre class="language-shell"><code class="language-shell"><span class="token builtin class-name">alias</span> <span class="token assign-left variable">conflicts</span><span class="token operator">=</span><span class="token string">"git status -sb | grep UU | awk '{print <span class="token variable">$2</span>}' | grep -v '\.min\.' | xargs code"</span></code></pre>
<p>That's great for things that are personal to me. This <code>conflicts</code> command is something I use, and it's identical for every project.</p>
<p>On the other hand, not all commands are identical between all projects. For example, if you're using Docker to build a local development environment that functions like your production environment, the containers and the commands to compose them together are going to differ from project to project; so having that alias live in my profile doesn't make as much sense. I'll need new aliases for every project, and keeping them in sync with the way the environment works is difficult. It would be great if there were a way to commit them to source control. 🤔</p>
<p>And bonus: making it part of the code repository means it's shared across the team and all team members can benefit from it.</p>
<blockquote>
<p>Obviously there's <a href="https://docs.docker.com/compose/">Docker-Compose</a>, but that will <em>only</em> compose a local development environment. This Makefile approach is useful beyond that and actually pairs well with Docker Compose. We use compose to compose a collection of containers as our local development environment, but we have a Makefile that we use to build and publish production containers to Amazon ECR, and deploy them on Amazon ECS, and Docker Compose can't do that. It also won't automate the little things like attaching to a shell session inside your Nginx container.</p>
</blockquote>
<h2 id="a-make-primer" tabindex="-1">A Make Primer</h2>
<p>This isn't going to be an article on the intricacies of Make and Makefiles (<a href="https://www.gnu.org/software/make/manual/html_node/index.html">learn more here</a>), but here's a quick Make primer.</p>
<p>I learned about it in my early programming classes as a tool that we could use to orchestrate compiling our C and C++ programs. It allows you to create "targets" (think scripts) that have a sequential list of commands to run, but they can also optionally depend on other targets <em>and on files</em>.</p>
<p>If compiling your program required a certain object (<code>foo.o</code>) file that itself had to be compiled, you could simply depend on that file <code>foo.o</code> and Make would build it for you if the file doesn't exist. How does it build that file? You create a target with the same name: <code>foo.o</code>.</p>
<pre class="language-makefile"><code class="language-makefile"><span class="token target symbol">myapp</span><span class="token punctuation">:</span> foo.o <span class="token comment">#depends on foo.o</span><br /> <span class="token comment"># building myapp executable</span><br /> cc -o myapp myapp.c<br /><br /><span class="token target symbol">foo.o</span><span class="token punctuation">:</span><br /> <span class="token comment"># building foo.o from foo.c</span><br /> cc -o foo.o foo.c</code></pre>
<p>You run <code>make myapp</code> and when all is said and done, your app should be compiled...</p>
<p>Make is super powerful, and writing this post reminded me that one of the few books I kept from college was <a href="https://amzn.to/3ccrtJp">Linux in a Nutshell</a>, which is kind of like the manpages of all of the most common Linux tools in printed form for quick reference, but a little better. I checked, and mine (<em>3rd edition, printed August 2000! I'm so old!</em>) does have a section on Make. A quick skim showed me a few things that I'm eager to learn more about, so I popped a bookmark in and left it on my desk to come back to later. (Narrator: He won't.)</p>
<h2 id="sharing-aliases-and-workflows-with-makefiles" tabindex="-1">Sharing aliases and workflows with Makefiles</h2>
<p>Cool, so now we understand the power of aliases to simplify complex commands and give them short and easily remembered names; and we're eager to share commands with our teammates. How can Make help with that?</p>
<p>Let's start with the goal of running a docker container, and building it first if necessary. We'll aim to use the command <code>make up</code> to start our environment.</p>
<pre class="language-makefile"><code class="language-makefile"><span class="token target symbol">up</span><span class="token punctuation">:</span><br /> docker run myapp<br /><br /><span class="token target symbol">build</span><span class="token punctuation">:</span><br /> docker build -t myapp .</code></pre>
<p>I've defined one target to start the container and one target to build it, but there's no dependency between them. How do you depend on something that doesn't create a physical artifact in Make? Well, I don't know if this is a good idea or not but I've had some success using hidden files to indicate things like build status:</p>
<pre class="language-makefile"><code class="language-makefile"><span class="token target symbol">up</span><span class="token punctuation">:</span> .myapp-built<br /> docker start myapp<br /><br /><span class="token target symbol">build</span><span class="token punctuation">:</span> .myapp-built<br /> <span class="token operator">@</span>echo Container built.<br /><br /><span class="token target symbol">.myapp-built</span><span class="token punctuation">:</span><br /> docker build -t myapp . && touch .myapp-built</code></pre>
<p>Here I'm using a hidden file <code>.myapp-built</code> to indicate that the container has been built. If I were to add a shortcut to delete the container for some reason, it should also delete that file to indicate to Make that the container doesn't exist any more. You'll also want to add <code>.myapp-built</code> to <code>.gitignore</code>.</p>
<p>Here's a slightly more thorough example showing targets to start, stop, build, and rebuild your container.</p>
<pre class="language-makefile"><code class="language-makefile"><span class="token target symbol">up</span><span class="token punctuation">:</span> .myapp-built<br /> <span class="token operator">@</span>docker run --rm -d myapp && touch .myapp-running<br /><br /><span class="token target symbol">down</span><span class="token punctuation">:</span> .myapp-running<br /> -docker stop myapp<br /> <span class="token operator">@</span>rm -f .myapp-running<br /><br /><span class="token target symbol">build</span><span class="token punctuation">:</span> .myapp-built<br /> <span class="token operator">@</span>echo Container built.<br /><br /><span class="token target symbol">rebuild</span><span class="token punctuation">:</span><br /> <span class="token operator">@</span>make down<br /> <span class="token operator">@</span>rm -f .myapp-built<br /> <span class="token operator">@</span>make build<br /><br /><span class="token target symbol">.myapp-running</span><span class="token punctuation">:</span><br /> <span class="token comment">#if you try to run `make down` when container isn't running, you'll be here</span><br /> <span class="token operator">@</span>touch .myapp-running<br /><br /><span class="token target symbol">.myapp-built</span><span class="token punctuation">:</span><br /> <span class="token operator">@</span>docker build -t myapp . && touch .myapp-built</code></pre>
<h2 id="what's-up-with-the-%40-and---prefixes%3F" tabindex="-1">What's up with the @ and - prefixes?</h2>
<p>Given:</p>
<pre class="language-makefile"><code class="language-makefile"><span class="token target symbol">hello</span><span class="token punctuation">:</span><br /> echo Hello, world.</code></pre>
<p>If you run: <code>make hello</code>, the output will be:</p>
<pre><code>echo Hello, world.
Hello, world.
</code></pre>
<p>Prefixing a command, such as <code>echo</code> with an <code>@</code> tells Make not to print the command to the output. With the same example as above, if we changed <code>echo</code> to <code>@echo</code>, the output would be:</p>
<pre><code>Hello, world.
</code></pre>
<p>The <code>-</code> prefix tells Make to ignore any errors that command might throw. For example, if you try to stop a container that's not running, Docker will throw an error. Normally Make would stop executing because of the error. But since we just want to make sure that it's not running and we don't care if it was already not running, we can ignore that error.</p>
<h2 id="is-there-a-better-way%3F" tabindex="-1">Is there a better way?</h2>
<p>This is the best way I've found to share automation shortcuts with my team. We've also used npm scripts to do similar work in the past, but I feel like they aren't quite as robust. Since they have to go into package.json, you end up having to jump through some hoops to make them work as one-liners that can live in a JSON string, and there's no baked-in dependency resolution.</p>
<p>I don't love that we have a bunch of <code>.dotfiles</code> hanging around to indicate automation statuses, but that's kind of the only drawback I've seen so far, and it's a small enough price to pay.</p>
<p>I don't know how well Make works on Windows, and of course it's going to require WSL, but hopefully developer machines are all on recent enough versions Windows to have WSL by now. 🤷♂️</p>
<p>If you have any better ideas, I'd love to hear them.</p>
Malicious Users in Express, Revisited2020-09-18T00:00:00Zhttps://adamtuttle.codes/blog/2020/malicious-users-in-express-revisited/<p><img src="https://adamtuttle.codes/img/2020/tarik-haiga-BxELNNMN88Y-unsplash.jpg" alt="Anonymous mask" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@tar1k?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Tarik Haiga</a> on <a href="https://unsplash.com/s/photos/anonymous?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>Back in August I wrote about <a href="https://adamtuttle.codes/blog/2020/rate-limiting-a-malicious-user-in-express/">how we added some rate limiting to an Express.js application to thwart someone using an online donation form -- ostensibly -- to verify stolen credit cards</a>. That worked ok for us, <em>for a while.</em></p>
<p>Eventually the attacker came back, and this time they were much more determined to accomplish their goal, even if it was going to take them a while. The problem with the brute-force protection that we added is that it has to have some leeway for legitimate users. It's not unheard of for a person to have a declined card or even two, so you have to allow at least a few retries without any cooldown period enforced.</p>
<p>Further, we found that our donation form is such a usable interface that school staff are using it to charge some credit card gifts submitted via paper forms. This makes it totally legitimate for a single person (ip address) to make 10 or more donations in a single day; so a per-day limit is out of the question.</p>
<p>Add to that the ease of browsing through the Tor network to change your IP constantly, and brute force protection alone was not enough to protect us. This attacker would sometimes disapepar for days at a time before returning to submit a burst of 20+ charges in a single morning, spaced out to avoid being throttled.</p>
<p>So here's what we changed.</p>
<p>To allow staff members to continue using our gift form as an easy way to digitize a paper gift we do have to reset the brute-force-request-counter after each successful charge, which at face value might seem like a bad thing (more opportunities for fraudulent charges), but doing so gave us the confidence to clamp down the other parameters of brute force prevention. We're still using a Fibonacci spiral cooldown, but it starts with a larger number and that makes it increase more rapidly. We also reduced the number of cooldown-free retries to just two.</p>
<p><img src="https://adamtuttle.codes/img/2020/Im_not_a_robot.jpg" alt="Screen shot of Google's infamous "I'm not a robot" checkbox CAPTCHA" /></p>
<p>More importantly, we added a CAPTCHA. We resisted this for a long time, in part because having 3rd party scripts on the page where someone is entering their credit card information is a big risk, <a href="https://www.pcisecuritystandards.org/">PCI</a>-wise. Ultimately it became clear to us that while there is some risk, there's simply no choice in the matter. Without a CAPTCHA there's no way to be certain that the request is coming from a human and not a bot; and the small PCI risk is well worth it to not have to clean up dozens of fraudulent/incomplete gifts every week as well as the potential hit to your gateway reputation. Gateways don't much care for you if you can't keep fraudulent activity away.</p>
<p>As tempting as it was to go down the rabbit hole of looking for signals to detect bot requests, we are not in the machine learning business and it was a smarter move to outsource that determination to someone who has the resources and the motivation to perfect it. Yes, we used <a href="https://developers.google.com/recaptcha/">Google's reCAPTCHA</a>.</p>
<p>We know that everyone is sick of training Google's self driving vehicle algorithms to detect motorcycles, stop lights, and crosswalks, so we were delighted to find that there's <a href="https://developers.google.com/recaptcha/docs/invisible"><strong>a version of reCAPTCHA that's invisible in most cases</strong></a>. It works like a regular captcha except that it stays invisible -- doesn't even require you to click the "I'm not a robot" checkbox -- unless it's unsure whether or not you're a bot.</p>
<p>For most of our users, they should never notice a difference. And for those that get the CAPTCHA, at least we know they're being served something accessible and that meets (is!) the industry standard for the test.</p>
<p>Mostly this whole post was just inspired by the fact that there is an invisible version of reCAPTCHA available. I had no idea until we resigned ourselves to the fact that we had no choice but to add a CAPTCHA, and now we'll probably be adding it in more locations to protect our customers from fraudulent charges and abusive data cleanup.</p>
<p>I love it when a moment of sadness can be turned into a moment of happiness.</p>
Getting Started with Taffy, 2020 Edition2020-09-10T00:00:00Zhttps://adamtuttle.codes/blog/2020/getting-started-with-taffy-2020/<p><img src="https://adamtuttle.codes/img/2020/will-o-GtYFwFrFbMA-unsplash.jpg" alt="A hot air balloon inflating" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@blnk_kanvas?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Will O</a> on <a href="https://unsplash.com/s/photos/getting-started?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>Recently a friend asked me for tips on getting started with <a href="https://taffy.io/">Taffy</a> -- my framework for building REST API's in CFML.</p>
<p>I sent him a link to the <a href="https://taffy.io/documentation.html#guides">guides</a> but with the caveat that those are pretty old by now (looks like most of them are from 2012-2014! 😬) but they should be at least mostly still relevant.</p>
<p>But the truth is, Taffy is built specifically to be dead simple to get on its feet, and capable of doing anything you need to do in a REST API. So here's a quick getting started brain dump, 2020 edition. It's written for someone with a passable understanding of how REST works who just needs to understand how to make Taffy do that work for them. If you're looking for more detail on exactly what REST is, why, how, etc, I wrote a book for you: <a href="https://restassuredbook.com/">REST Assured</a>.</p>
<h2 id="library-location-%2F-mapping" tabindex="-1">Library location / mapping</h2>
<p>Either place the <code>taffy</code> folder in your web root (no danger in that), or create a mapping for <code>/taffy</code> to wherever you have it available.</p>
<h2 id="application.cfc" tabindex="-1">Application.cfc</h2>
<p>Yes, your API needs its own <code>Application.cfc</code>. <em><strong>The Taffy Way</strong></em> is for the API to live in its own folder separate from your user-facing application if you have one. For example: <code>/api/v1/</code>.</p>
<p>Your Application.cfc should extend <code>taffy.core.api</code>, and have a name. That's all that's <em>required</em> but you'll probably want to do more:</p>
<pre class="language-js"><code class="language-js">component <span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.api"</span> <span class="token punctuation">{</span><br /> <span class="token keyword">this</span><span class="token punctuation">.</span>name<span class="token operator">=</span><span class="token string">"my_awesome_api"</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Taffy implements <code>onApplicationStart()</code>, <code>onRequestStart()</code>, <code>onRequest()</code>, and <code>onError()</code> so if you need to add your own code to these methods, make sure to call the appropriate super function:</p>
<pre class="language-js"><code class="language-js">component <span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.api"</span> <span class="token punctuation">{</span><br /> <span class="token keyword">this</span><span class="token punctuation">.</span>name<span class="token operator">=</span><span class="token string">"my_awesome_api"</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">function</span> <span class="token function">onRequestStart</span><span class="token punctuation">(</span><span class="token parameter">targetPath</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> request<span class="token punctuation">.</span>foo <span class="token operator">=</span> <span class="token string">"bar"</span><span class="token punctuation">;</span><br /> <span class="token keyword">super</span><span class="token punctuation">.</span><span class="token function">onRequestStart</span><span class="token punctuation">(</span>arguments<span class="token punctuation">.</span>targetPath<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>Whether you call the super function before or after your code additions is up to you. Think about what the framework does at that moment in time, what you're trying to add, and what the best order of operations should be. It is conceivable to have code before and after for a variety of purposes.</p>
<p>The next thing to know about Application.cfc is that you may add some configuration in <a href="http://docs.taffy.io/3.2.0#variablesframework-settings">variables.framework</a> -- though again, this is entirely optional. That link shows the default configuration, so you only need to include/change bits that you want to be different. Each setting is loaded/defaulted individually so that you can use the object shorthand syntax <code>variables.framework = { ... };</code> to specify only the settings you want to override, and the rest will still use their defaults.</p>
<blockquote>
<p><strong>Homework:</strong> You're probably pretty quickly going to want to be able to intercept and inspect an incoming REST request before Taffy passes it on to your handler code. Think about things like rate limiting, API key validation, etc. Things that you'll want to apply broadly to most if not all of your API surface area. To do that, Taffy allows you to implement an <code>onTaffyRequest()</code> method in your Application.cfc. Your response from that method determines whether or not the request continues; and you can also modify the request before it does, if you wish. Just make a note of this fact, and come back to <a href="http://docs.taffy.io/3.2.0#ontaffyrequest">read the docs on onTaffyRequest()</a> when the time is right.</p>
</blockquote>
<h2 id="index.cfm" tabindex="-1">index.cfm</h2>
<p>You need an <code>index.cfm</code> to go along with your <code>Application.cfc</code>. It doesn't matter what's in it; it will never be executed -- blank is fine. It just has to exist so that your HTTP server (e.g. Apache) will route the incoming request to CFML to handle.</p>
<h2 id="%3Cbreakpoint%3E" tabindex="-1"><breakpoint></h2>
<p>You now have a functioning API. It can't <em>do anything</em> but that's beside the point. It accepts REST calls and will return 404 for most reasonable requests because it doesn't know about any routes to things. But it's a REST API, and if you're using all of the framework defaults, you did that in as little as 3 lines of code spanning 2 files. That's pretty great, right?</p>
<p>Ok, let's get back to work...</p>
<h2 id="a-quick-note-on-vocabulary" tabindex="-1">A quick note on vocabulary</h2>
<p>Half of what REST is about is URL's, so routing is an important part of any REST framework. This is more of a general REST thing than a Taffy thing, but you should familiarize yourself with the concept of Collections and Members. A collection is multiple records of the same type (e.g. from the same database table), and a member is one of those records. <code>/widgets</code> is a collection, while <code>/widgets/42</code> is a member.</p>
<h2 id="adding-resources-%2F-routes" tabindex="-1">Adding resources / routes</h2>
<p>One of Taffy's primary goals is to give you a way to easily provide something resembling traditional REST URL's. Let's start by creating a widgets collection resource. First, create a <code>resources/</code> folder adjacent to your Application.cfc and index.cfm, and in it place a CFC that extends <code>taffy.core.resource</code>:</p>
<pre class="language-js"><code class="language-js">component<br /><span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.resource"</span><br />taffy_uri<span class="token operator">=</span><span class="token string">"/widgets"</span><br /><span class="token punctuation">{</span><br /><br /> <span class="token keyword">function</span> <span class="token function">get</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">representationOf</span><span class="token punctuation">(</span><span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select * from widgets"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>Here you can see we've also added component metadata named <code>taffy_uri</code>, and a method named <code>get</code>. In that <code>get</code> method, we make use of another function <code>representationOf</code>. Let's explain what these things do:</p>
<p>The component metadata <code>taffy_uri</code> tells taffy that this CFC is responsible for handling all requests to the url <code>index.cfm/widgets</code>. So if your domain was <a href="http://example.com/">example.com</a> and you used the suggested <code>/api/v1</code> folder structure, the full URL would look like this: <code>http://example.com/api/v1/index.cfm/widgets</code>. I wish that the index.cfm didn't have to be there, but it does. (You can <a href="https://github.com/atuttle/Taffy/wiki/URL-Rewrite-Rule-Examples">use URL rewriting to get rid of it</a>, but that's an exercise left for the reader.) No matter the verb, any request that comes in to <code>/widgets</code> will be handled by this CFC.</p>
<p>The reason our example method was named GET was so that it will respond to the HTTP GET method. To respond to POST you implement a method named POST. To respond to PUT you make a method named PUT, and DELETE for DELETE. You should never have to implement OPTIONS, Taffy handles these requests for you.</p>
<p><strong>As you may already know, REST is mostly about HTTP verbs and URLs, so here we've already figured out the basics.</strong></p>
<ul>
<li>GET returns existing data</li>
<li>POST inserts new data</li>
<li>PUT updates existing data or inserts new when you are including the new primary key value in the payload</li>
<li>DELETE deletes data</li>
</ul>
<p>Of course the implementation details of each of those are left to you.</p>
<p>Of the three new concepts introduced in the last code sample, all that remains unexplained is <code>representationOf()</code>. This is a helper method provided for you by Taffy to help with separation of concerns. Note that I returned a raw query resultset, not a string representation of those results. The <code>representationOf()</code> method -- or you can use <code>rep()</code> for shorthand if you like! -- does the work of passing your native response data off to a tool that will serialize it appropriately for the client.</p>
<h3 id="serialization" tabindex="-1">Serialization</h3>
<p>By default Taffy uses a serialization class that uses the native <code>serializeJson()</code> functionality. If you recall, that method turns CFML queries into JSON, but JSON that's kind of <em>"special."</em> You may want to convert that query into an array of structs before you return it. This is such a common use case that Taffy includes a helper method to do this for you: <code>queryToArray</code>, and unlike raw <code>serializeJson(query)</code> the column name text-case you used in your query will be preserved -- your object key names won't be in ALL UPPER CASE.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">get</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">rep</span><span class="token punctuation">(</span><span class="token function">queryToArray</span><span class="token punctuation">(</span><span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">'select * from widgets'</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<blockquote>
<p><strong>Homework:</strong> queryToArray also takes a 2nd function to act as a callback, which makes it a really powerful utility to have on your toolbelt! When you're ready, <a href="http://docs.taffy.io/3.2.0#querytoarray">read more about it here</a>.</p>
</blockquote>
<blockquote>
<p><strong>Homework:</strong> Taffy completely supports other serialization options by allowing you to implement your own serializer class. And deserializers too, for non-json request body payloads. You can read more about those here: <a href="http://docs.taffy.io/3.2.0#custom-serializers">Serializers</a>, <a href="http://docs.taffy.io/3.2.0#custom-deserializers">Deserializers</a>.</p>
</blockquote>
<h3 id="what-about-input%3F" tabindex="-1">What about input?</h3>
<p>There are a few ways of providing request data to a Taffy request:</p>
<ol>
<li>URI Tokens</li>
<li>Query String Params</li>
<li>Body params</li>
</ol>
<p>They are evaluated in the order listed above, and if there are ever conflicts, the last one in wins. So if you have a request with a URI token named foo, a query string param named foo, and a body param named foo, the handler will get one foo argument, and its value will be the one from the body.</p>
<p>Taffy takes care of parsing all of the request data for you and makes it available as arguments to your handler functions.</p>
<h4 id="uri-tokens" tabindex="-1">URI Tokens</h4>
<p>Specify a token in your <code>taffy_uri</code> like <code>{this}</code>. So for our widgets <strong>member</strong> CFC, we might write the following code:</p>
<p><code>index.cfm/widgets/42</code></p>
<pre class="language-js"><code class="language-js">component<br /><span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.resource"</span><br />taffy_uri<span class="token operator">=</span><span class="token string">"/widgets/{id}"</span> <span class="token punctuation">{</span><br /><br /> <span class="token keyword">function</span> <span class="token function">get</span><span class="token punctuation">(</span> <span class="token parameter">id</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//id == 42</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>URI tokens are required in order to match -- there's no such thing as an optional token. For optional arguments use query string params. For example, a filter:</p>
<p><code>index.cfm/widgets?filter=blue</code></p>
<pre class="language-js"><code class="language-js">component<br /><span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.resource"</span><br />taffy_uri<span class="token operator">=</span><span class="token string">"/widgets"</span> <span class="token punctuation">{</span><br /><br /> <span class="token keyword">function</span> <span class="token function">get</span><span class="token punctuation">(</span> <span class="token parameter">filter</span> <span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//filter == "blue"</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>Body params are also optional. By default, Taffy is capable of handling both JSON and <code>application/x-www-form-urlencoded</code> request payloads. If the request body is JSON, Taffy expects it to be an object. The keys of that object will be passed as arguments to your handler function. If it is not an object -- say it's an array instead -- then Taffy passes the entire body as the argument <code>_body</code>.</p>
<h2 id="wrapping-up" tabindex="-1">Wrapping Up</h2>
<p>That covers almost all of the core of REST API functionality. So what's left?</p>
<ul>
<li>Status codes</li>
<li>Response headers</li>
</ul>
<p>Both of these are easily added to any response by chaining a method onto your existing response:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">return</span> <span class="token function">rep</span><span class="token punctuation">(</span>someData<span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">withStatus</span><span class="token punctuation">(</span><span class="token number">201</span><span class="token punctuation">,</span> <span class="token string">'Created'</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">withHeaders</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token string-property property">'x-foo'</span><span class="token operator">:</span> <span class="token string">'bar'</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>I know I've written a lot of words here, but if you look carefully, we've written a scary-small amount of code to get a lot of functionality out of Taffy. That was one of the driving forces behind its development!</p>
<p>All of the information in this blog post is covered in even more detail in the <a href="https://docs.taffy.io/">Taffy documentation</a>, but hopefully the cherry-picked topics and organizational style of this blog post helped you get on your feet. That was the goal!</p>
<blockquote>
<img src="https://restassuredbook.com/assets/img/book-3d.png" alt="Cover art for REST Assured" class="hero" width="100" style="color:black;float:right; margin-bottom:0" />
<strong>If you're looking to learn even more about REST then I suggest you take a look at my book, <a href="https://restassuredbook.com/">REST Assured</a>!</strong><br /><br />It teaches you all of the practical foundational knowledge about REST without getting bogged down in academic restraints that aren't useful in the real world.
<div style="clear:right"></div>
</blockquote>
Happy 10th Birthday, Taffy!2020-08-24T00:00:00Zhttps://adamtuttle.codes/blog/2020/happy-10th-birthday-taffy/<p><img src="https://adamtuttle.codes/img/2020/annie-spratt-M20ylqCzSZw-unsplash.jpg" alt=""Happy Birthday" spelled out in lit candles" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@anniespratt?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Annie Spratt</a> on <a href="https://unsplash.com/s/photos/birthday?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>As programmers we often look at code we wrote even as recently a few months ago in disgust. Sometimes we just have to do something distasteful to get things done on time or on budget; or sometimes we've learned a lot since the code was written; and sometimes we can't event explain it.</p>
<p>Taffy is TEN years old today 🍰🥳 (Technically, yesterday, but it was the weekend and who's counting?) and it still ranks as one of the pieces of software that I am most proud of in my career. I created it because I was fascinated by doing REST well in CFML, and not particularly thrilled with any of the options that were available at the time. None of them were <em>bad</em>, but I had learned some things in recent years and saw that there was potential to build something better. So I did.</p>
<p>It now represents a ton of personal growth and success for me, successful collaboration with the community at large, and many opportunities to continue branching out into other related interests. I got to speak at several conferences, and <a href="https://restassuredbook.com/">I wrote a book about REST</a>, and that wouldn't have happened without the success of Taffy.</p>
<h2 id="happy-birthday-taffy!" tabindex="-1">Happy Birthday Taffy!</h2>
<p>To celebrate Taffy's 10 year birthday, I finally got around to restoring the database backup from my last website so that I can resurrect articles from it! I think I've found and restored every post about Taffy from the last 10 years, from <a href="https://adamtuttle.codes/blog/2010/Taffy-A-Restful-Framework-for-ColdFusion/">the original announcement in 2010</a> through <a href="http://iq.localhost.tools:8000/blog/2015/Taffy-3-1-0-RC1/">Taffy 3.1.0-RC1</a> in 2015. I guess I never posted after <a href="https://github.com/atuttle/Taffy/releases/tag/v3.1.0">the final 3.1.0 release</a> or for <a href="https://github.com/atuttle/Taffy/releases/tag/v3.2.0">3.2.0</a>. 🤷♂️</p>
<p>All of those old posts are now listed on <a href="https://adamtuttle.codes/blog">the full blog index</a>, and now's a good time to look for them because the early years don't have much more than Taffy content at the moment. I'll get around to restoring the other old posts too, but with limited time and the birthday and all, it seemed like a good way to narrow the scope for phase 1 of the restoration project. (I started blogging in 2007 and wrote 48 entries that year! 😬)</p>
<p>Also... This doesn't really have any useful meaning (I think?) but I thought it might be fun to go back and look at just how much of that old code is still hanging around. How many lines of 10 year old code are there still being used today in Taffy? I'll drop a note about the methodology for getting these numbers at the bottom of this article. For now, let's just enjoy some numbers.</p>
<h3 id="taffy.core.api" tabindex="-1">taffy.core.api</h3>
<p><img src="https://adamtuttle.codes/img/2020/taffy-10-counts-api.png" alt="A chart showing the line counts of taffy.core.api by date they were written." /></p>
<p>Aside from a few spikes here and there, the line count is pretty evenely distributed from early on in the repo through approximately August of 2015. It tapers off a bit after that because we've been pretty stable ever since. The 3.1 release came after, but includes a lot of the later changes in that time range (3.0 was released in November of 2014).</p>
<p>To reiterate, this is not a count of new lines over time, this is a count of the number of lines that were written on the given date that remain unmodified in the project today. That it is so front-loaded is also something I take pride in: We got it "right" early on.</p>
<h3 id="taffy.core.resource" tabindex="-1">taffy.core.resource</h3>
<p><img src="https://adamtuttle.codes/img/2020/taffy-10-counts-resource.png" alt="A chart showing the line counts of taffy.core.resource by date they were written." /></p>
<p>There are far fewer lines of code in taffy.core.resource, so I'm not surprised that the counts aren't as high; but again it looks like the story is mostly the same. Early efforts were pretty solid and some additions throughout 2013 have proven useful enough to stand the test of time.</p>
<h3 id="dashboard.cfm" tabindex="-1">dashboard.cfm</h3>
<p><img src="https://adamtuttle.codes/img/2020/taffy-10-counts-dashboard.png" alt="A chart showing the line counts of taffy/dashboard/dashboard.cfm by date they were written." /></p>
<p>The dashboard got <a href="https://github.com/atuttle/Taffy/commit/4811a932086a03100d045eb111ae7744b25ad220#diff-54694bb9609e53b9bc660d0d7f14b3ae">a ground-up rewrite in 2013</a> so its current iteration doesn't have anything older than that.</p>
<h3 id="documentation" tabindex="-1">Documentation</h3>
<p>I'm super proud of everything about Taffy, and that extends equally to <a href="http://docs.taffy.io/">our documentation</a>. It's not perfect (what docs are?), but we've tried really hard to make sure that everything is explained and that it's reasonably well organized. We even keep <a href="http://docs.taffy.io/2.0.0">documentation for older versions</a> still available and usable! That's more than I can say for a lot of open source projects that just burn it all down and start over with new documentation when they start a new major version of the library.</p>
<p>Combined between all versions, we currently have more than 16,400 lines of markdown documentation. There is some repetition when things don't change between versions, but I still think this is a wonderful thing because it maintains what's relevant to any given user no matter what version of Taffy they're using.</p>
<h3 id="contributors" tabindex="-1">Contributors</h3>
<p>None of this would have happened if people hadn't:</p>
<ol>
<li>Seen value in the code I was sharing,</li>
<li>Tried it for themselves, and</li>
<li>Cared enough to file bug reports or feature requests, share it with their peers, and in some cases, even contribute code and documentation back to the project.</li>
</ol>
<p>As of right now we've had 42 people contribute to the code repo, and 7 of them also contributed to the documentation. I am so grateful and humbled for all of your participation over the last DECADE!</p>
<p>Thank you!! 🥳 🤗 🍾</p>
<h3 id="methodology%2C-or%2C-where-did-these-numbers-come-from%3F" tabindex="-1">Methodology, or, Where did these numbers come from?</h3>
<p>While they aren't perfect, I was able to extract something close by using <code>git blame</code> to get the date that each line of each file was last modified. I combined this with some *nix CLI tools to get counts by date:</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">git</span> blame dashboard.cfm <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{print $4}'</span> <span class="token operator">|</span> <span class="token function">sort</span> <span class="token operator">|</span> <span class="token function">uniq</span> <span class="token parameter variable">-c</span></code></pre>
<p>I copied this data into a few Excel sheets and used Excel's charting features to generate the charts.</p>
<p>The documentation line count was generated using this command in the <code>src</code> folder of the Docs repo:</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">wc</span> <span class="token parameter variable">-l</span> *.md <span class="token operator">|</span> <span class="token function">grep</span> <span class="token string">'\d\.\d\.\d\.md'</span> <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{print $1}'</span> <span class="token operator">|</span> <span class="token function">sum</span></code></pre>
<p>The contributor counts are provided directly from GitHub.</p>
Book Updates and Livestreams2020-08-19T00:00:00Zhttps://adamtuttle.codes/blog/2020/book-updates-and-livestreams/<h2 id="%F0%9F%93%98%E2%98%9D%EF%B8%8F%F0%9F%93%85-book-updates" tabindex="-1">📘☝️📅 Book updates</h2>
<p>The last update I published for <a href="https://restassuredbook.com/">REST Assured</a> was in February of 2019. It's time for another. There's at least one topic that I intentionally glossed over back in 2015 (JWT) because it was so new and not yet widely used that there was no way to know if it would stand the test of time. To be honest, I didn't know much about it yet, either. JWT has proven itself useful in the last 5 years, so it's time to give it fair representation in the book.</p>
<p>There are other areas that could use some attention, but while I have your attention I should ask: Have you read <strong>REST Assured</strong> and wished it covered some other aspect of REST or of API's in general? Did I not dive deep enough into a topic, or did my explanation fall short of connecting with you? Leave a comment and let me know, and I just might make your wish come true in the next edition.</p>
<h2 id="%F0%9F%93%BA-%F0%9F%92%BB-more-than-just-a-book%3F!" tabindex="-1">📺 💻 More than just a book?!</h2>
<p>I also want to create additional resources to offer bundled with the book.</p>
<p>If you're like me, you learn better from (or is it pay better attention to?) video content than books. I think that a set of screencasts that explain the concepts from the book in a video format could be really useful.</p>
<p>I'm also thinking about adding a sample application. It would be something simple and straight forward, like a todo list, but it would implement (almost?) every concept from the book for easy reference.</p>
<p>I haven't figured out the precise details yet, but I'll be absolutely certain that anyone who purchased the book before the bonus resource bundles were available can get a huge discount on the upgrade.</p>
<h2 id="%F0%9F%93%B9-%E2%8F%AF%EF%B8%8F-livestreams%2C-too!" tabindex="-1">📹 ⏯️ Livestreams, too!</h2>
<p>I've taken an interest in livestream video lately, too. I'd like to stream the entire process of updating the book and creating the screencasts and sample application. The streams are sure to be less refined than the final product, but you can get a lot of this bonus content for free by tuning in live or watching the stream recordings.</p>
<p>Plus you get to watch me flub the script and mock me in the stream chat.</p>
<p><a href="https://www.youtube.com/channel/UCOeYypSs8QoqX6AgA5eqGEg?sub_confirmation=1">Subscribe to my YouTube channel</a> (and be sure to click the bell that appears next to the subscribe button) to get notifications when I go live. I don't have a specific schedule in mind yet, but I'm thinking I might set aside one or two nights per week to stream & work.</p>
Rate Limiting a Malicious User in Express.js2020-08-18T00:00:00Zhttps://adamtuttle.codes/blog/2020/rate-limiting-a-malicious-user-in-express/<p><img src="https://adamtuttle.codes/img/2020/tarik-haiga-BxELNNMN88Y-unsplash.jpg" alt="Anonymous mask" /></p>
<p>Here's an interesting problem! Someone started attempting to submit many donations through an online form we run for one of our customers. <em>Sounds like a pretty ok problem to have, right?</em> Sure, when I explain it naïvely enough. Let's look closer.</p>
<p>To limit our <a href="https://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Security_Standard">PCI DSS</a> exposure, we use a credit card processing service that tokenizes the card details and handles communicating with payment gateways for us. The short version is that while you do enter your credit card details on our web app, they never hit our servers or end up in our logs and instead we get a token that we can use to initiate a payment.</p>
<p>We noticed a user submitting our donation form's final action, where we accept the credit card token along with the gift details, repeatedly. Not just repeatedly, but rapidly. So fast that they were obviously automating the requests and not simply clicking "submit payment" over and over. With the same token; which would not be representative of genuine transactions. A genuine user would tokenize once and submit that token once, and we should in theory never see that token used again.</p>
<p>You might be thinking, "This seems odd, but they're trying to give you money. What's so bad about that? 🤔"</p>
<p>At least, that was <em>my</em> first reaction. But I've already spilled the beans; we think this was a malicious action.</p>
<p>There's no way to be sure, but here's what we think is going on. This person has aquired a list of stolen credit card numbers, and wants to find out if any of them still work. If they can attempt to make a $5 gift with each one, and a few of the cards work, then they've identified the useful numbers from the list and they can then put those cards to use to make bigger purchases elsewhere without having set off any alarms at the company that would be fulfilling that larger purchase. Better to tip off the recipient of a dozen $5 transactions than a dozen $800 transactions.</p>
<p>That's our theory, anyway.</p>
<p>How did we notice this? Well, we take a belt and suspenders approach to monitoring and reporting anomalous behavior. Of course the requests were all logged, but nobody could possibly keep up with the logs and also make forward progress on development goals. The logs are more for postmortem analysis. In addition to HTTP logs, though, we log all exceptions including failed payments, and in certain mission critical areas (like the one where people give you money out of the goodness of their hearts), every failure is worth looking at to at least confirm it was user error (credit limit reached, incorrect card number, etc). When something unexpected happens, we set off alarms. Literally. In addition to pushing notifications into Slack, we also alert our on-call developer.</p>
<p>We use OpsGenie (#notsponsored) to manage our on-call rotation, alert the on-call developer, and escalate if that developer doesn't deal with the issue in a timely manner. One of the must-have features was a way to wake the on-call person up, no matter what. OpsGenie has an app that, when given correct permissions, can "poke through" your phone's Do Not Disturb mode, set the volume to max, and play an alarm sound of your choosing. I'm currently using <a href="https://www.youtube.com/watch?v=CQeezCdF4mk">sad trombone</a>.</p>
<p>One Sunday evening we started getting a bunch of alarms. A slow but annoying trickle at first. Then right around midnight, our bad actor decided to open the flood gates. I have never heard so much sad trombone in my life. They triggered a continuous alarm that simply could not be ignored, even if no real harm was coming to our app or data.</p>
<p>With no other choice, I decided to shut down the online giving form for the customer under attack. (Mark this down as another reason to split your monolith up into microservices! Making that happen in a monolith would have required making some code modifications at midnight, groggy from interrupted sleep, with sad trombone constantly playing in my ears. Not ideal.)</p>
<p>After getting a few more hours of sleep, I gathered the team and we put our heads together on the right solution to get this particular brand of jerk-store to buzz off. We discussed adding rate limiting at the <abbr title="Web Application Firewall">WAF</abbr> layer, but ultimately decided it was the wrong tool for the job. The rate limiting controls there start with defaults like 1,000 requests per 5 minute window, which might be fine if you're dealing with general rate limiting, but aren't well suited for a targeted attack on one URI. Even if we set it to a relatively low value like 100 requests per 5 minutes, that's still allowing another request every 3 seconds or so, indefinitely.</p>
<p>That would not be much of an improvement over continuous sad trombone.</p>
<p>Fortunately, this microservice is built on Express.js, which means it has a huge community of free and open source plugins waiting to be taken advantage of. We eventually settled on <a href="https://www.npmjs.com/package/express-brute">Express-Brute</a>, which allowed us to apply a rate limit specifically to the one route in question, and configure it exactly how we wanted.</p>
<p>We went with a fibonacci-sequence-based cooldown after a couple of "free" retries. Leaving some wiggle room for genuine donors to make mistakes seemed like the right approach. Our fibonacci cooldown starts at a couple of minutes, and very quickly spirals out to a full hour cooldown, with something like a 5-8 hour memory. The theoretical maximum number of requests our bad actor could make in 24 hours is now far fewer than even just the number of attempts they made between 12:00 and 12:01 on that Monday morning.</p>
<p>I was able to add all of that with only 19 new lines of code to import, setup, and use the plugin. The plugin itself is very small, and its dependencies are pretty dang limited considering how significantly it just saved our bacon!</p>
<p>The service has been back online for quite some time now and we've seen the malicious actor make a couple more attempts, but the change seems to have done its job and sent them off to find another credit card form to abuse.</p>
<p>I never imagined I'd write code to reduce the frequency of someone giving us money, but here we are.</p>
How to Get Started Folding@Home for COVID19 Research2020-04-10T00:00:00Zhttps://adamtuttle.codes/blog/2020/folding-at-home-for-covid19/<p><img src="https://adamtuttle.codes/img/2020/carolina-garcia-tavizon-Ay7Nkvc49ag-unsplash.jpg" alt="Colorful origami paper cranes" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@karock12?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Carolina Garcia Tavizon</a> on <a href="https://unsplash.com/s/photos/origami?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>Well this is nuts, huh? <em>*gestures broadly*</em></p>
<p>I want to keep this short and sweet to not waste any of your time, so let's just jump right into it, right? Most of us probably have <em>at least</em> 1 computer running 24x7 these days. That includes a lot of time where it's doing little, if anything, of value. Before the pandemic hit, I had heard of <a href="https://foldingathome.org/">Folding@Home</a> but never got involved. When some coworkers found out that some COVID19 research was being done on the platform, most of us jumped at the chance to help.</p>
<p>If you don't already know, it's a way to let your computer contribute to distributed protein-folding research.</p>
<h2 id="the-short%2C-short-version-(for-nerds)" tabindex="-1">The short, short version (for nerds)</h2>
<p>If you're on a mac and you have docker installed, you can probably run <a href="https://gist.github.com/atuttle/4c4b3c9d4f3cec46d59015a20750af24">this makefile</a> (that I wrote) to get started. It literally takes only a couple of minutes to download the dependencies and start everything up, and then you're online and folding. You could stop there, or you could <a href="https://foldingathome.org/start-folding/">install the controller app</a> which will allow you to tweak your power-consumption settings, watch the logs of what your process is doing, and have a bit more control over it.</p>
<h2 id="for-everyone-else" tabindex="-1">For everyone else</h2>
<p>If you don't have a mac, or docker, or you don't know how to use a makefile, no worries. Just <a href="https://foldingathome.org/start-folding/">download the installer from here</a> and you'll be up and running just as fast.</p>
<h2 id="everything-else" tabindex="-1">Everything else</h2>
<p>By default after you start your folding@home process for the first time, you'll be configured to run work units for "any disease", which is currently the best way to contribute to COVID19 (there's no dedicated disease preference for it); so that's great. You also start folding anonymously. You can sign up for an account and get a passkey (<a href="https://apps.foldingathome.org/getpasskey">here</a>) to take credit for your work, and doing so also allows you to contribute to a team. If for some reason you want to contribute to our team, our team id is <code>250123</code> 🤘</p>
<p>Personally, I am running this on both my work computer and my HTPC, both at full-power constantly, and aside from the fan noise and thermal output, I haven't noticed a difference in my computer's speed; which is pretty great! I'm helping out and my work and daily usage of the computer is unaffected! Granted, <a href="https://adamtuttle.codes/blog/2019/building-a-hackintosh-2019/">I have a pretty highly-spec'd computer</a>, but still...</p>
<p>There are leaderboards, if you're into that sort of thing. With 4 of us at <a href="https://www.alumniq.com/">AlumnIQ</a> folding, it took <a href="https://stats.foldingathome.org/team/250123">us</a> about 10-14 days to break into the top 4% of teams. The top tier teams have many orders of magnitude more members and resources they can bring to bear, so our chances of ever catching them are basically nil, but it's fun anyway. We have a small slack celebration any time any of us hits a points milestone.</p>
Multi-Tenant Express.js Redis Sessions2020-03-20T00:00:00Zhttps://adamtuttle.codes/blog/2020/multi-tenant-express-redis-sessions/<p>Yesterday at work we did what we usually do: Looked at a problem, came up with a solution idea, and when that didn't work we adjusted course until we reached what felt like success. But...</p>
<p>It's working, but it <em>feels like something that maybe nobody has done before, <strong>and maybe for good reason?</strong></em></p>
<p>I'd like to tell you about what we did, and why we did it, so that if you see some reason this is not a good idea you can let me know. And hey, maybe this is a good approach and we'll solve your problem too. 🤷♂️</p>
<h2 id="the-problem" tabindex="-1">The Problem</h2>
<p>We need to start a new application using Node, Express.js, and hook it up to Redis for sessions. So far this is pretty normal. Here's how you'd do that for an average app.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> express <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'express'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> session <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'express-session'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> RedisStore <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'connect-redis'</span><span class="token punctuation">)</span><span class="token punctuation">(</span>session<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> redisClient <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'redis'</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">createClient</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">host</span><span class="token operator">:</span> <span class="token function">getSetting</span><span class="token punctuation">(</span><span class="token string">'keyStoreHost'</span><span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">const</span> app <span class="token operator">=</span> <span class="token function">express</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line">app<span class="token punctuation">.</span><span class="token function">use</span><span class="token punctuation">(</span></span><br /><span class="highlight-line"> <span class="token function">session</span><span class="token punctuation">(</span><span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">secret</span><span class="token operator">:</span> <span class="token string">'keyboard-cat'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">key</span><span class="token operator">:</span> <span class="token string">'cookie-name'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">proxy</span><span class="token operator">:</span> <span class="token string">'true'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">resave</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">saveUninitialized</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">store</span><span class="token operator">:</span> <span class="token keyword">new</span> <span class="token class-name">RedisStore</span><span class="token punctuation">(</span><span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">client</span><span class="token operator">:</span> redisClient<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">ttl</span><span class="token operator">:</span> <span class="token number">60</span> <span class="token operator">*</span> <span class="token number">60</span><span class="token punctuation">,</span> <span class="token comment">//1 hour</span></span><br /><span class="highlight-line"> <span class="token literal-property property">prefix</span><span class="token operator">:</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">session:</span><span class="token template-punctuation string">`</span></span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>So here's where it gets complicated. We want our application to be <a href="https://en.wikipedia.org/wiki/Multitenancy">multi-tenant</a>, and to identify which customer owns the session for the user making the request, which we can tell from the hostname. We include the customer id, among other things, in our session identifiers: <code>sessions:customer-abc:etc:</code>. Partly to allow session sharing between tech stacks and applications, we would very much like to continue taking this approach.</p>
<blockquote>
<p>I suppose it should be noted here that if we were willing to relinquish on that one requirement and have all customers share one pool of sessions, this entire problem kind of goes away. It would make it that much more difficult to find the right redis key on the rare occasion we want to dig up someone's session contents to debug something, and it would require modifying other applications to change their session prefixes, and I am a little bit worried about the idea of session id collisions, but it's possible that it could go fine.</p>
</blockquote>
<p>Our first thought was to (my favorite word: "just") wrap the <code>session()</code> middleware with a custom function that inspects the request to set a variable that could be consumed via closure in the session middleware. Unfortunately, it's not that simple. Calling <code>session()</code> here <em>returns</em> a middleware function, and by that point the value in the <code>prefix</code> option is saved in memory and not editable.</p>
<p>I briefly considered the idea of forking <strong>Connect-Redis</strong> or <strong>Express-Session</strong> (or both) to always use a callback to get the <code>prefix</code> value, but quickly realized that we don't want to take on any ongoing maintenance of those modules. I guess if it turns out that the callback is a good idea, we may reach out to those developers and ask if they're ok with a PR that adds it as an option. But for now, that idea is tabled.</p>
<p>So here's where we landed:</p>
<h2 id="the-solution-(for-now%3F)" tabindex="-1">The Solution (for now?)</h2>
<p>Step 1: A new middleware to identify the customer from the hostname.</p>
<pre class="language-js"><code class="language-js">app<span class="token punctuation">.</span><span class="token function">use</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span> res<span class="token punctuation">,</span> next</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>req<span class="token punctuation">.</span>hostname <span class="token keyword">in</span> <span class="token constant">MAP</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> req<span class="token punctuation">.</span><span class="token constant">CUSTOMER</span> <span class="token operator">=</span> <span class="token constant">MAP</span><span class="token punctuation">[</span>req<span class="token punctuation">.</span>hostname<span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> <span class="token function">next</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span><br /> <span class="token keyword">throw</span> <span class="token keyword">new</span> <span class="token class-name">Error</span><span class="token punctuation">(</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">Unrecognized hostname: </span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>req<span class="token punctuation">.</span>hostname<span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>Step 2: A new custom middleware-generator that will create new <strong>Express-Session</strong> and associated <strong>RedisStore</strong> instances —one for each customer, created only once, at startup— and then return a middleware that will proxy the request to the appropriate session-middleware instance based on <code>req.CUSTOMER</code>. I know that sounds complicated and weird when written out, but hopefully it will make more sense written out in code form.</p>
<p>So we start with our <code>app.use(session(...))</code> and replace that with a new middleware call:</p>
<pre class="language-js"><code class="language-js">app<span class="token punctuation">.</span><span class="token function">use</span><span class="token punctuation">(</span><span class="token function">dynamicSessions</span><span class="token punctuation">(</span><span class="token string">'appName'</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>That <code>dynamicSessions</code> middleware accepts an app name (another part of what we put in the session prefix), and will need to do all of the stuff I described above, so here we go.</p>
<p>This is the implementaiton of <code>dynamicSessions()</code>. I'll discuss each chunk as we go, and then give the entire thing in one block again at the end.</p>
<p>First, we need to bring in our dependencies, including a list of customers, and create a hash table where we'll store the instantiated and ready to use session middlewares.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> session <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'express-session'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> RedisStore <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'connect-redis'</span><span class="token punctuation">)</span><span class="token punctuation">(</span>session<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> redisClient <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'redis'</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">createClient</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">host</span><span class="token operator">:</span> <span class="token function">getSetting</span><span class="token punctuation">(</span><span class="token string">'keyStoreHost'</span><span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token comment">//for this purpose, just assume this is an array of customer id's:</span></span><br /><span class="highlight-line"><span class="token comment">// ['customer-A', 'customer-B', etc]</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> getCustomers <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'./our-customers'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token comment">// a hash table to store references to the middlewares we generate</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> customerSessionMiddleware <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>Then, we need two config objects for the options for express-session and RedisStore that won't be changing between our instances; and a function that we can use to generate the various middlewares based on the two things that will be different between them, the customer id and the app name. I hope that the two objects are self explanatory. The function uses a <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax">spread expression</a> to expand those objects into a new object, and add on the only thing that's different between each customer, the RedisStore instance and its prefix attribute.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> SessionMiddlewareConfig <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">secret</span><span class="token operator">:</span> <span class="token string">'keyboard-cat'</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">resave</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">saveUninitialized</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> <span class="token literal-property property">key</span><span class="token operator">:</span> <span class="token string">'cookie-name'</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">const</span> RedisStoreConfig <span class="token operator">=</span> <span class="token punctuation">{</span><br /> <span class="token literal-property property">client</span><span class="token operator">:</span> redisClient<span class="token punctuation">,</span><br /> <span class="token literal-property property">ttl</span><span class="token operator">:</span> <span class="token number">60</span> <span class="token operator">*</span> <span class="token number">60</span> <span class="token comment">//1 hour in seconds</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br /><span class="token keyword">const</span> <span class="token function-variable function">configureMiddleware</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">cust<span class="token punctuation">,</span> app</span><span class="token punctuation">)</span> <span class="token operator">=></span><br /> <span class="token function">session</span><span class="token punctuation">(</span><span class="token punctuation">{</span><br /> <span class="token operator">...</span>SessionMiddlewareConfig<span class="token punctuation">,</span><br /> <span class="token literal-property property">store</span><span class="token operator">:</span> <span class="token keyword">new</span> <span class="token class-name">RedisStore</span><span class="token punctuation">(</span><span class="token punctuation">{</span><br /> <span class="token operator">...</span>RedisStoreConfig<span class="token punctuation">,</span><br /> <span class="token literal-property property">prefix</span><span class="token operator">:</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">sessions:</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>cust<span class="token punctuation">.</span><span class="token function">toLowerCase</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">:</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>app<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">:</span><span class="token template-punctuation string">`</span></span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>And lastly, we need to create the function that we're going to export. It will accept the app name, generate all of the necessary middlewres (one for each customer) and store them in the hash table we previously created; and then it will return a new function that Express will hold onto. That function will inspect the request for the <code>req.CUSTOMER</code> key we created with our other middleware, use that to select the appropriate session middleware from the hash table, and proxy the invocation to that middleware.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> <span class="token function-variable function">dynamicSessions</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">app</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token comment">//load all session stores up once at startup</span><br /> <span class="token function">getCustomers</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">forEach</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">cust</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> customerSessionMiddleware<span class="token punctuation">[</span>cust<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">configureMiddleware</span><span class="token punctuation">(</span>cust<span class="token punctuation">,</span> app<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">return</span> <span class="token keyword">function</span> <span class="token function">dynamicSessionMiddleware</span><span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span> res<span class="token punctuation">,</span> next</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">const</span> customerMiddleware <span class="token operator">=</span> customerSessionMiddleware<span class="token punctuation">[</span>req<span class="token punctuation">.</span><span class="token constant">CUSTOMER</span><span class="token punctuation">]</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> <span class="token function">customerMiddleware</span><span class="token punctuation">(</span>req<span class="token punctuation">,</span> res<span class="token punctuation">,</span> next<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br />module<span class="token punctuation">.</span>exports <span class="token operator">=</span> dynamicSessions<span class="token punctuation">;</span></code></pre>
<p>So then once again, here it is all together:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> session <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'express-session'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> RedisStore <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'connect-redis'</span><span class="token punctuation">)</span><span class="token punctuation">(</span>session<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> redisClient <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'redis'</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">createClient</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">host</span><span class="token operator">:</span> <span class="token function">getSetting</span><span class="token punctuation">(</span><span class="token string">'keyStoreHost'</span><span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token comment">//for this purpose, just assume this is an array of customer id's:</span></span><br /><span class="highlight-line"><span class="token comment">// ['customer-A', 'customer-B', etc]</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> getCustomers <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'./our-customers'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token comment">// a hash table to store references to the middlewares we generate</span></span><br /><span class="highlight-line"><span class="token keyword">const</span> customerSessionMiddleware <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">const</span> SessionMiddlewareConfig <span class="token operator">=</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">secret</span><span class="token operator">:</span> <span class="token string">'keyboard-cat'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">resave</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">saveUninitialized</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">key</span><span class="token operator">:</span> <span class="token string">'cookie-name'</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">const</span> RedisStoreConfig <span class="token operator">=</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">client</span><span class="token operator">:</span> redisClient<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">ttl</span><span class="token operator">:</span> <span class="token number">60</span> <span class="token operator">*</span> <span class="token number">60</span> <span class="token comment">//1 hour in seconds</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">const</span> <span class="token function-variable function">configureMiddleware</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">cust<span class="token punctuation">,</span> app</span><span class="token punctuation">)</span> <span class="token operator">=></span></span><br /><span class="highlight-line"> <span class="token function">session</span><span class="token punctuation">(</span><span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token operator">...</span>SessionMiddlewareConfig<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">store</span><span class="token operator">:</span> <span class="token keyword">new</span> <span class="token class-name">RedisStore</span><span class="token punctuation">(</span><span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token operator">...</span>RedisStoreConfig<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">prefix</span><span class="token operator">:</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">sessions:</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>cust<span class="token punctuation">.</span><span class="token function">toLowerCase</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">:</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>app<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">:</span><span class="token template-punctuation string">`</span></span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"><span class="token keyword">const</span> <span class="token function-variable function">dynamicSessions</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">app</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token comment">//load all session stores up once at startup</span></span><br /><span class="highlight-line"> <span class="token function">getCustomers</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">forEach</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">cust</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> customerSessionMiddleware<span class="token punctuation">[</span>cust<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">configureMiddleware</span><span class="token punctuation">(</span>cust<span class="token punctuation">,</span> app<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">function</span> <span class="token function">dynamicSessionMiddleware</span><span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span> res<span class="token punctuation">,</span> next</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">const</span> customerMiddleware <span class="token operator">=</span> customerSessionMiddleware<span class="token punctuation">[</span>req<span class="token punctuation">.</span><span class="token constant">CUSTOMER</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">customerMiddleware</span><span class="token punctuation">(</span>req<span class="token punctuation">,</span> res<span class="token punctuation">,</span> next<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line">module<span class="token punctuation">.</span>exports <span class="token operator">=</span> dynamicSessions<span class="token punctuation">;</span></span></code></pre>
<p>I've got this deployed in a sandbox, and it seems to be working well. Load testing doesn't show any memory leaks, and everything seems to work as expected.</p>
<p>So, internet, what boogeymen are lurking in the shadows here? Why is this a terrible idea?</p>
February Goals Check-In2020-03-03T00:00:00Zhttps://adamtuttle.codes/blog/2020/february-goals-check-in/<p>At the end of 2019 <a href="https://adamtuttle.codes/blog/2019/my-smart-goals-for-2020/">I posted a couple of goals for 2020</a>. It's been two months and I'm at the checkpoint for my first milestone, so I thought I would check in on how I've been doing.</p>
<h2 id="goal-1%3A-body-fat-reduction" tabindex="-1">Goal 1: Body fat reduction</h2>
<p>I started eating a Ketogenic diet in November 2019, and as a result had to either torture myself through the holidays by watching everyone else eat delicious treats and not partake, or I had to find a way to get my sweet fix. I found some recipies for things like cookies and frosting using keto-friendly sugar substitutes, and, let's be honest here, I also started to play a little fast and loose with the 20g/day carbohydrate limit. My limit of 20 became 21, and eventually 22, and 23... and before you know it, I can't seem to stay in ketosis at all.</p>
<p>To make matters <em>"worse"</em>, we had a family vacation scheduled for the end of January. A cruise, to be specific. If you've never been on a cruise, let me explain how the food works: You can eat as much as you want, at just about any time of day, but almost everything has carbs. Also, there's a formal-ish dinner every night with fancy desserts to choose from; all included for the cost of the cruise. Why should I pay for fancy food and not eat it?!</p>
<p>If you were super disciplined, I suppose you could make it work.</p>
<p>But I'm not, and I couldn't.</p>
<p>And I didn't.</p>
<p>In fact, in the days leading up to the cruise, and knowing that I would be at least temporarily abandoning my keto diet, I chose to quit early.</p>
<p>At the time that I quit eating keto, I had managed to work my way down to 19.8% BF already! I was <em>way ahead of schedule.</em> This was mid-January and I had already passed my goal for March 1st. And then I went back to my junkfood mid-day snacks and carb-loaded meals, and only a week after getting back from the cruise, I was back up to 21.2% BF. <strong>Ugh!</strong></p>
<p>That was February 8th, and it was quite the wake-up call. I've been back on Keto since, and trending back down. I'm happy to report that I did <em>just barely</em> manage to hit my 20% BF on March 1st goal.</p>
<p>This goal also had other aspects to help me reach that metric. Not missing many gym days, and hit my macros consistently. Well, I pretty much blew the macros thing up for a while there, but I'm back on the wagon now.</p>
<p>I've also missed more gym days than I was anticipating, but overall I'm really happy with my gym attendance. I got sick once, I chose to do a career day presentation that didn't leave enough time in the day to work out, and, yeah, a couple of times I was too lazy. BUT! Most of the time I don't struggle to find the motivation to get into the gym, and almost always I feel great when I leave the gym. I'm not seeing much muscle mass added yet, but I feel healthier, and that's a big part of why I want to go, too.</p>
<p>I also managed to find time to work out during the cruise! I only missed one workout and in my defense, traveling -- especially with children! -- is exhausting! I missed my cardio day on the day we spent traveling back home.</p>
<p>I've found that I really love doing squats, probably because I can already squat a lot more weight than I thought would be in my reach. In fact, my training program recently had me re-evaluate my 1 rep max on all of my primary lifts, and this happened:</p>
<p><a href="https://twitter.com/AdamTuttle/status/1233389017418412038">https://twitter.com/AdamTuttle/status/1233389017418412038</a></p>
<p>So I guess let's say I get a solid B+ on this one. I can do better, but nothing to be ashamed of here.</p>
<p>I learned just how effective Keto can be for me, and I also learned that just because the rule of thumb recommendation for beginner keto is 20g of carbs per day doesn't mean I need to aim to finish every day at 19.9. If I aim for 10-15, there's wiggle room for those times when I need a couple of spoonfulls of <a href="https://www.thekitchn.com/aldi-keto-ice-cream-review-23003042">keto ice cream</a> to get me through the day.</p>
<h2 id="goal-2%3A-christmas-present-for-my-wife" tabindex="-1">Goal 2: Christmas present for my wife</h2>
<p>Well. This one is going to have to end up being an excused absence if I don't come up with a new idea. I managed to bring casual conversation just close enough to the topic of the project I wanted to make for her to find out that it's not something she wants at all. Back to the drawing board!</p>
<h2 id="goal-3%3A-build-my-cnc-and-learn-how-to-use-it" tabindex="-1">Goal 3: Build my CNC and learn how to use it</h2>
<p>Step 1: Build it. Check! ✔️ (Mostly? I ran out of t-track so there's a small area of the wasteboard that's incomplete while I wait for more to come in the mail. But c'mon! Close enough for credit, right?!)</p>
<p><img src="https://adamtuttle.codes/img/2020/cnc-build.jpg" alt="" /></p>
<p>I have definitely started learning. I've already broken one router bit by setting up a carve that was too aggressive. Failures lead to the best learnings!</p>
<h2 id="goal-4%3A-wean-down-to-no-soda" tabindex="-1">Goal 4: Wean down to no soda</h2>
<p>This one might also require some re-thinking. Ideally I would avoid soda completely. I'm sure most doctors would agree that would be the best thing. But on the keto diet I personally find the complete lack of sweets to be crazy. I don't know how people do it. I've been drinking Diet Mountain Dew (#notsponsored) for a really long time, and since it's 0 calorie (or close enough to round down) and 0 carbs, and it satisfies my sweet tooth, I'm willing to flex on this.</p>
<p>I still believe that reducing soda consumption would be beneficial to my health, but if I can keep soda levels where they are and it enables me to hit weight and body fat goals that would be otherwise (nearly?) unattainable, is that an acceptable trade? I think it might be.</p>
<h2 id="unofficial-goals%3A" tabindex="-1">Unofficial goals:</h2>
<p>There are also a few things that I didn't include in my official list, but that I was trying to stay on top of as if they were official goals anyway. In particular, gratitude journaling and being diligent with backups of my Hackintosh.</p>
<p>Long story short, I've been terrible at doing either one of them. I keep meaning to get back to both of them, but intention doesn't check the box. Only action can do that.</p>
Lucee Error: Unrecognized Id Type : Double -> java.lang.Double2020-02-27T00:00:00Zhttps://adamtuttle.codes/blog/2020/lucee-error-unrecognized-id-type-double-java-lang-double/<p>Today's adventure in the ways in which ORM in Lucee is <em>a little off</em> got started when I tried to use ORM to create a new record. That entity uses an identity column as its primary key.</p>
<pre class="language-js"><code class="language-js">property<span class="token punctuation">;</span><br />name <span class="token operator">=</span> <span class="token string">'accountId'</span><span class="token punctuation">;</span><br />type <span class="token operator">=</span> <span class="token string">'numeric'</span><span class="token punctuation">;</span><br />fieldtype <span class="token operator">=</span> <span class="token string">'id'</span><span class="token punctuation">;</span><br />generator <span class="token operator">=</span> <span class="token string">'identity'</span><span class="token punctuation">;</span></code></pre>
<p>The problem only reared its head when attempting to insert a new row into this table using ORM.</p>
<p><img src="https://adamtuttle.codes/img/2020/lucee-identity-col-error.png" alt="Screen shot of error message, "unrecognized id type : double -> java.lang.Double"" /></p>
<p class="photo-byline">unrecognized id type : double -> java.lang.Double</p>
<p>Fortunately, when I mentioned this at our daily standup meeting, a coworker remembered that he had run into this in the past. I imagine that he found a fix at the time by googling and finding an answer in a discussion group or something, so I'm glad to have had him to help me! The fix isn't obvious! See if you can spot it:</p>
<pre class="language-js"><code class="language-js">property<span class="token punctuation">;</span><br />name <span class="token operator">=</span> <span class="token string">'accountId'</span><span class="token punctuation">;</span><br />type <span class="token operator">=</span> <span class="token string">'numeric'</span><span class="token punctuation">;</span><br />fieldtype <span class="token operator">=</span> <span class="token string">'id'</span><span class="token punctuation">;</span><br />ormtype <span class="token operator">=</span> <span class="token string">'integer'</span><span class="token punctuation">;</span><br />generator <span class="token operator">=</span> <span class="token string">'identity'</span><span class="token punctuation">;</span></code></pre>
<p>The fix was adding the attribute <code>ormtype="integer"</code>. I'm told that <code>ormtype="int"</code> -- while <a href="https://cfdocs.org/cfproperty">technically valid</a> -- won't fix it. I tried it in one location and didn't have any problems, but I figured it was worth mentioning, you know, just in case.</p>
<p>While we're here, can we take a moment to acknowledge how strange it is to have an attribute named <code>ormType</code> that exists specifically for use with the primary key? The name seems general enough to be applicable to any column. But right <a href="https://helpx.adobe.com/coldfusion/developing-applications/coldfusion-orm/define-orm-mapping/map-the-properties.html">from the ACF docs</a>:</p>
<blockquote>
<p>Used to specify the data type of the primary key. If data type is not set and the ORM setting useDBForMapping=true, then the ormtype is determined by inspecting the database. The different data types that are supported by ColdFusion are specified in the ORM data types.</p>
</blockquote>
<p>I can't find any docs on <code>ormType</code> in the Lucee docs, which is ironic considering that this attribute wasn't necessary in our code on ACF but is necessary on Lucee.</p>
Hackintosh Part III: Daily Driver2020-02-25T00:00:00Zhttps://adamtuttle.codes/blog/2020/hackintosh-iii-daily-driver/<blockquote>
<p>This is part three in an ongoing series where I document my experiences building my first Hackintosh.</p>
<ul>
<li><a href="https://adamtuttle.codes/blog/2019/building-a-hackintosh-2019/">Part one: My motivations, and hardware I purchased</a></li>
<li><a href="https://adamtuttle.codes/blog/2019/building-a-hackintosh-2019/">Part two: Hardware build complete, OS Install "feature complete"</a></li>
</ul>
</blockquote>
<p>It's been almost a year since I embarked on the journey of attempting to replace my Mid-2014 model Macbook Pro with a custom built Hackintosh, so we're way overdue for an update.</p>
<p>Spoiler alert: The reason this update seems so overdue is that it's been going so well that I've been head-down on work and personal stuff. So that's great news.</p>
<p>With that in mind, let's start with...</p>
<h2 id="the-bad-stuff" tabindex="-1">The Bad Stuff</h2>
<h3 id="i-can't-install-any-os-updates" tabindex="-1">I can't install any OS updates</h3>
<p>That primarily means security patches. I'm not extremely worried about this, because this machine doesn't leave my house (it's not a laptop), I'm the only user, and I can and do easily avoid worms spread through email, messenger clients, etc. I know that doesn't protect everything, and <em>that's</em> why I've left a little bit of room for worry. I do worry, but obviously not so much that it's worth paying the Apple hardware tax.</p>
<p>And for what little it's worth, there are <em>some</em> updates (cough, Catalina, cough) that are simply not acceptable for me yet. Some apps don't support Catalina yet, and I need those apps more than I need a random OS upgrade.</p>
<p>I'll talk more about OS updates in <strong>The Good Stuff</strong>, below.</p>
<h3 id="usb-is-a-little-bit-flaky" tabindex="-1">USB is a little bit flaky</h3>
<p>Not flaky as in things stop working at random times, but that certain USB ports work and others don't, and my USB hub doesn't always agree with OSX. For example, I bought a <a href="https://amzn.to/3a4RwQB">tiny USB micro-sd card reader</a>, and plugging that into my USB hub only mounts the memory card as a volume maybe 1 out of 10 times. I can either use a non-hub USB port to get that working, or boot into Windows to offload my files. Since offloading my files usually entails dumping them into cloud storage, it doesn't much matter where I do it. The only time it's been a nuissance is when I want to take footage from my GoPro and edit it immediately. A small price that I'm more than happy to continue paying in exchange for the other benefits.</p>
<p>I know there are various hackintosh USB drivers available to try out, but to emphasize how little this problem bothers me, I haven't found it worth my time to try fixing the problem.</p>
<h3 id="the-os-thinks-that-there's-an-apple-webcam%2Fmicrophone-available" tabindex="-1">The OS thinks that there's an Apple webcam/microphone available</h3>
<p>As a remote worker, 90%+ of my meetings are web conferences using my webcam and microphone. I'm using a USB webcam to act as both of those, but that doesn't stop the OS from expecting the built-in webcam and microphone that it would have on any modern Apple hardware. It hasn't been an issue once I got things configured initially, but each app that uses the webcam and/or microphone will need to be told to use the USB options, not the "built-in" (missing) options.</p>
<h3 id="i-can't-run-any-virtual-machines" tabindex="-1">I can't run any virtual machines</h3>
<p>I don't use VM's much. As a webdev on OSX, it would be nice to be able to test something out on Edge or IE11 occasionally, and <a href="https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/">Microsoft provides free VM's for exactly that purpose</a>. Alas, booting a virtualbox VM on my hackintosh instantly crashes it. I've managed not to suffer from this, but it has been annoying once or twice in the last year.</p>
<h2 id="the-good-stuff" tabindex="-1">The Good Stuff</h2>
<p>This is fine! And I actually mean that, not in the dog-in-flaming-room sense.</p>
<p><img src="https://adamtuttle.codes/img/2020/this-is-fine.jpg" alt="comic of dog sitting in a burning building, saying "this is fine."" /></p>
<p>I'm just as productive as ever. Arguably more, with a faster CPU, dedicated GPU, and 4x more RAM. Aside from being unable to run virtual machines and the feeling of unease from not being able to install OS updates, as a daily driver, I've reached 100% of my productivity goals.</p>
<h3 id="os-updates-are-possible%2C-just-not-as-easy" tabindex="-1">OS Updates are possible, just not as easy</h3>
<p>After each OSX update, the Hackintosh community is hard at work making it possible to install from scratch with that new latest version. So when I get far enough behind, and/or an upgrade becomes actually necessary, or when Amazon can start delivering free time in boxes with smiles on them with free next-day shipping, then I can give it a shot.</p>
<p>And here's the great thing about this whole hackintosh process. The safest path forward when attempting these changes is to clone your OS SSD onto a spare, try the upgrade, and if it doesn't work out, clone back. Worst case scenario, you've burned a couple of hours and lost nothing <a href="https://adamtuttle.codes/blog/2020/time-is-all-you-have/">but time</a> in the process.</p>
<p>If it works out, then you install the apps that you use that aren't cloud services, and get back to work. Maybe you burn a day in that process? And let's be honest, who among us doesn't revel in the thought of moving into a freshly installed and pristine OS?</p>
<h3 id="i've-also-got-my-gaming-computer" tabindex="-1">I've also got my gaming computer</h3>
<p>One of the not-so-secret bonuses of building a hackintosh was that I would wind up with a computer capable of dual booting into Windows and, gasp, being able to play modern games.</p>
<p>My dual-boot Windows installation is not without its own quirks (perhaps for another time), but it serves its purpose just fine. It's been many years since I've had a Windows gaming computer at my disposal, and I've had a lot of fun getting into Steam for the first time. (I know, right?! 😱) I switched to Mac as my primary/only computer around 2008-2010, and while that's been great for work it's also meant I completely lost touch with PC gaming.</p>
<h2 id="what-would-it-take-to-move-to-a-new-machine%3F" tabindex="-1">What would it take to move to a new machine?</h2>
<p>I mentioned before that OS updates/upgrades are relatively painless. First of all, in 2020, the cloud is pretty great.</p>
<p>For work purposes, I need: Slack (which I can use in a browser tab in a pinch), 1Password, Dropbox (mostly for 1Password team vault sharing), my VPN client, VS Code, node/npm, our various project git repositories, Docker, iTerm, and a browser or two. Chrome does a nice job of syncing my extensions and bookmarks. I have a private git repo with my dotfiles, which include things like my oh-my-zsh config and a crapload of command aliases and shortcuts that I've created and made habits out of over the years.</p>
<blockquote>
<p>As much as <a href="https://adamtuttle.codes/blog/2019/making-firefox-usable-on-osx/">I would prefer to use Firefox as my daily browser</a>, it just can't keep up with Chrome, even in basic page rendering speeds. When your job is managing websites, a second here and a half second there add up really quickly. I had to abandon my morals and go back to Chrome. 😭</p>
</blockquote>
<p>That's enough to get by on for work in a pinch, and I think I could get that all on its feet in a half day or less. So what nice-to-haves does that leave?</p>
<p>Here are a bunch of things that aren't critical path, but that make life as a developer or general OSX usage happier for me: <a href="https://github.com/Clipy/Clipy#readme">Clipy</a>, <a href="https://evernote.com/products/skitch">Skitch</a>, <a href="https://matthewpalmer.net/rocket/">Rocket</a>, <a href="https://www.macbartender.com/">Bartender</a>, <a href="https://flexibits.com/fantastical">Fantastical</a>, <a href="https://sequelpro.com/">Sequel Pro</a>, Slack ("native" app), Sourcetree, <a href="https://www.alfredapp.com/">Alfred</a>, <a href="http://www.derlien.com/">Disk Inventory X</a>, <a href="https://pqrs.org/osx/karabiner/">Karabiner Elements</a>, <a href="https://magnet.crowdcafe.com/">Magnet</a>, <a href="http://getmedis.com/">Medis</a>, Trello, and VLC.</p>
<p>And these things are entirely for fun: <a href="https://obsproject.com/">OBS</a> (for streaming), Final Cut Pro, and <a href="https://apps.apple.com/us/app/gif-brewery-3-by-gfycat/id1081413713">GIF Brewery</a>.</p>
Lucee and ORM: Worth the Pain?2020-02-19T00:00:00Zhttps://adamtuttle.codes/blog/2020/lucee-and-orm/<p>I've written previously about how my company is in the process of <a href="https://adamtuttle.codes/blog/2019/from-coldfusion-to-lucee/">migrating our main app from Adobe ColdFusion to Lucee</a>. This post is an update, several months into that process.</p>
<p>Not unlike that first post, our biggest problem area in the migration since then has been ORM. Since then we've accomplished the primary goal of <em>the application starts and you can do some basic browsing</em>, and we're now deep into the process of <em>test every aspect of every feature, making sure they still work as expected</em>, which is... tedious.</p>
<p>The most recent problem we've been running into is that Lucee completely chokes if you've got a transaction that contains both an ORM read/write and a simple SQL query. There are different failure modes, but the worst is when the ORM usage comes first:</p>
<pre class="language-js"><code class="language-js">transaction <span class="token punctuation">{</span><br /> <span class="token keyword">var</span> x <span class="token operator">=</span> <span class="token function">entityLoadByPk</span><span class="token punctuation">(</span><span class="token string">"Foo"</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> x<span class="token punctuation">.</span><span class="token function">setBar</span><span class="token punctuation">(</span><span class="token string">"foo bar"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">entitySave</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select 1"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p><img src="https://adamtuttle.codes/img/2020/lucee-orm-error.png" alt="Screen shot of lucee error message. Content is repeated below." /></p>
<p>The error message thrown is, "this feature is not supported". (Firstly... thanks for the great error message! 👍)</p>
<p>You can ignore the fact that the SQL statement in the screen shot above is not <code>select 1</code>, matching my code snippet above. It doesn't matter for our purposes here. Any query will do.</p>
<p>We are, of course, <a href="https://luceeserver.atlassian.net/browse/LDEV-1564">not the first team to run into this issue</a>. In that Lucee bug report, others who found this before us suggested prepending the contents of any affected transaction with a simple query like the one seen in the above screen shot. We soon realized that the even-simpler <code>select 1;</code> was equally effective.</p>
<p>However, this alone doesn't make everything work. With the above code block updated like the following, Lucee no longer throws an exception, yay!</p>
<pre class="language-js"><code class="language-js">transaction <span class="token punctuation">{</span><br /> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select 1"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">var</span> x <span class="token operator">=</span> <span class="token function">entityLoadByPk</span><span class="token punctuation">(</span><span class="token string">"Foo"</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> x<span class="token punctuation">.</span><span class="token function">setBar</span><span class="token punctuation">(</span><span class="token string">"foo bar"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">entitySave</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select 1"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>It's great that no exception is thrown, but sadly it has stopped persisting the included ORM mutation at all, and is performing this failure <strong>silently</strong>. ORM reads seem to work fine as long as an SQL query is run first, but no mutations are persisted. <em>Not cool!</em></p>
<p>Even though Lucee have been silent on the bug report since May of 2018, others in the community have been helping each other out in the comments. One suggestion was that you could add a manual <code>ORMFlush()</code> after your transaction to get Lucee to do what it should have already done, and commit your ORM mutations. That did seem to work, but it made me curious about other potential problems; so I threw together a quick repro case to isolate the issue and allow me to test different theories.</p>
<pre class="language-js"><code class="language-js">component <span class="token punctuation">{</span><br /><br /> <span class="token keyword">function</span> <span class="token function">afterQuery</span><span class="token punctuation">(</span><span class="token parameter">rc</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> transaction <span class="token punctuation">{</span><br /> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select 1"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">var</span> o <span class="token operator">=</span> <span class="token function">entityLoadByPK</span><span class="token punctuation">(</span><span class="token string">"Pledge"</span><span class="token punctuation">,</span> <span class="token number">6000</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">var</span> existing <span class="token operator">=</span> o<span class="token punctuation">.</span><span class="token function">getHome_PostalCode</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">?</span><span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">;</span><br /> o<span class="token punctuation">.</span><span class="token function">setHome_PostalCode</span><span class="token punctuation">(</span> existing<span class="token operator">+</span><span class="token number">1</span> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">entitySave</span><span class="token punctuation">(</span>o<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">var</span> t <span class="token operator">=</span> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select home_postalCode as x from GiftPledge where pledgeId = 6000"</span><span class="token punctuation">)</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span><br /> <span class="token function">writeDump</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">before</span><span class="token operator">:</span> existing<span class="token punctuation">,</span> <span class="token literal-property property">expected</span><span class="token operator">:</span> existing<span class="token operator">+</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token literal-property property">actual</span><span class="token operator">:</span> t<span class="token punctuation">,</span> <span class="token literal-property property">pass</span><span class="token operator">:</span> t <span class="token operator">==</span> existing<span class="token operator">+</span><span class="token number">1</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> abort<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">function</span> <span class="token function">afterQueryWithFlush</span><span class="token punctuation">(</span><span class="token parameter">rc</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> transaction <span class="token punctuation">{</span><br /> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select 1"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">var</span> o <span class="token operator">=</span> <span class="token function">entityLoadByPK</span><span class="token punctuation">(</span><span class="token string">"Pledge"</span><span class="token punctuation">,</span> <span class="token number">6000</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">var</span> existing <span class="token operator">=</span> o<span class="token punctuation">.</span><span class="token function">getHome_PostalCode</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">?</span><span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">;</span><br /> o<span class="token punctuation">.</span><span class="token function">setHome_PostalCode</span><span class="token punctuation">(</span> existing<span class="token operator">+</span><span class="token number">1</span> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">entitySave</span><span class="token punctuation">(</span>o<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token function">ormFlush</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">var</span> t <span class="token operator">=</span> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select home_postalCode as x from GiftPledge where pledgeId = 6000"</span><span class="token punctuation">)</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span><br /> <span class="token function">writeDump</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">before</span><span class="token operator">:</span> existing<span class="token punctuation">,</span> <span class="token literal-property property">expected</span><span class="token operator">:</span> existing<span class="token operator">+</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token literal-property property">actual</span><span class="token operator">:</span> t<span class="token punctuation">,</span> <span class="token literal-property property">pass</span><span class="token operator">:</span> t <span class="token operator">==</span> existing<span class="token operator">+</span><span class="token number">1</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> abort<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>Here, I'm grabbing a random row of test data from our database via ORM, incrementing its postal code column as if it were some sort of counter (it's just test data and I just needed to see if mutations were persisted), and saving it. The <code>afterQuery</code> function doesn't have an <code>ORMFlush()</code> and the dump shows that the test fails: ORM thinks it's making the change (no exception), but it doesn't get persisted to the database.</p>
<p>The <code>afterQueryWithFlush</code> function is the same thing, but adds an <code>ORMFlush()</code> after the transaction ended. In this version, the mutation is persisted. So while that's less than ideal, at least we have a simple work-around.</p>
<p>An alternative work-around worth mentioning is to split ORM actions and SQL actions into separate transactions, if you can. If you can't, <code>ORMFlush()</code> is the only option we've found.</p>
<p>But this made me ask myself: If the ORM Session is getting flushed, does Lucee open a new session for the rest of the request to use? So I added another test:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">doubleWithFlush</span><span class="token punctuation">(</span><span class="token parameter">rc</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> transaction <span class="token punctuation">{</span><br /> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select 1"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">var</span> o <span class="token operator">=</span> <span class="token function">entityLoadByPK</span><span class="token punctuation">(</span><span class="token string">"Pledge"</span><span class="token punctuation">,</span> <span class="token number">6000</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">var</span> existing <span class="token operator">=</span> o<span class="token punctuation">.</span><span class="token function">getHome_PostalCode</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">?</span><span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">;</span><br /> o<span class="token punctuation">.</span><span class="token function">setHome_PostalCode</span><span class="token punctuation">(</span> existing<span class="token operator">+</span><span class="token number">1</span> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">entitySave</span><span class="token punctuation">(</span>o<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token function">ormFlush</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">var</span> t <span class="token operator">=</span> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select home_postalCode as x from GiftPledge where pledgeId = 6000"</span><span class="token punctuation">)</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span><br /> <span class="token function">writeDump</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">before</span><span class="token operator">:</span> existing<span class="token punctuation">,</span> <span class="token literal-property property">expected</span><span class="token operator">:</span> existing<span class="token operator">+</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token literal-property property">actual</span><span class="token operator">:</span> t<span class="token punctuation">,</span> <span class="token literal-property property">pass</span><span class="token operator">:</span> t <span class="token operator">==</span> existing<span class="token operator">+</span><span class="token number">1</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> transaction <span class="token punctuation">{</span><br /> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select 1"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">var</span> o2 <span class="token operator">=</span> <span class="token function">entityLoadByPK</span><span class="token punctuation">(</span><span class="token string">"Pledge"</span><span class="token punctuation">,</span> <span class="token number">6000</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">var</span> existing2 <span class="token operator">=</span> o2<span class="token punctuation">.</span><span class="token function">getHome_PostalCode</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">?</span><span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">;</span><br /> o2<span class="token punctuation">.</span><span class="token function">setHome_PostalCode</span><span class="token punctuation">(</span> existing2<span class="token operator">+</span><span class="token number">1</span> <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">entitySave</span><span class="token punctuation">(</span>o2<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token function">ormFlush</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">var</span> t2 <span class="token operator">=</span> <span class="token function">queryExecute</span><span class="token punctuation">(</span><span class="token string">"select home_postalCode as x from GiftPledge where pledgeId = 6000"</span><span class="token punctuation">)</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span><br /> <span class="token function">writeDump</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">before</span><span class="token operator">:</span> existing2<span class="token punctuation">,</span> <span class="token literal-property property">expected</span><span class="token operator">:</span> existing2<span class="token operator">+</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token literal-property property">actual</span><span class="token operator">:</span> t2<span class="token punctuation">,</span> <span class="token literal-property property">pass</span><span class="token operator">:</span> t2 <span class="token operator">==</span> existing2<span class="token operator">+</span><span class="token number">1</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> abort<span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>This is basically the same test as above, except that I'm repeating the mutation step a second time, after an <code>ORMFlush()</code>. Fortunately, this test passes just fine. That's a relief!</p>
<p>I haven't thought of any other problems the necessity of ORMFlush might cause us, but if any come up I'll be sure to write about them here.</p>
Time Is All You Have2020-02-18T00:00:00Zhttps://adamtuttle.codes/blog/2020/time-is-all-you-have/<p>To paraphrase a few things I heard on a podcast this week: time is all that you have; nobody knows how much time they'll get; and divided attention disappoints everyone involved.</p>
<p>This is a really hard lesson for me. I struggle with having too much ambition. When I start to enjoy consuming something, I start to want to become a producer.</p>
<p>I have discussed the idea of starting a podcast with more than one friend. I have dabbled in making and posting videos to youtube and occasionally lust at the idea of being a youtube personality in one niche or another (for me, it's usually woodworking or skydiving videos...), and I am constantly flirting with the idea of making some money from my woodworking hobby, but haven't yet.</p>
<p>That's just the first three things that come to mind. I've flirted with streaming video games on twitch (with the thought of involving my kids in that), and considered buying or building a sawmill and producing & selling lumber. I'm sure my HOA will frown on the noise, "eyesore" (even if I think it's beautiful), and other aspects of me having a big sawmill in my back yard. Some ideas are easier to let go than others. Chances of me having a sawmill in my yard are dangerously close to zero, which makes it hurt less to think of it as not even an option. (Maybe after I retire, if we move somewhere a little more rural?)</p>
<p>On the other hand, if I were to put some effort into it, I'm sure that I could make some money selling furniture or cutting boards or something; or I could build an audience for a podcast or videos. The ideas that could actually work are harder to let go.</p>
<p>This is not new: When I was a teenager I wanted to be a pro skateboarder (because I loved watching the x-games) even though I couldn't ollie to save my life; I wanted to play the guitar and piano (who doesn't at some point?) because I listen to a ton of music and often find it very meaningful. All my life I've had to fight the urge to pick up more hobbies. More to the point, I've occasionally chosen to actively quit some hobbies so that I could dedicate more time to others. At some point I decided that I no longer wanted to dedicate the time to practicing the guitar so I sold all of my guitars and amps and gave up the dream of becoming a rock star. It was hard to do, but once the artifacts were gone I felt a great sense of relief.</p>
<p>When you're young, time is abundant and money is hard to come by. As you get older, free time becomes more and more scarce and hopefully money becomes a little bit more attainable. I would go to great lengths today to get the free time I had when I was 16, and I'm sure that 16 year old me would have gone to great lengths to have access to the discretionary money I have available now.</p>
<p>Not at all to disparage them, but I'm sure that moving into the chapter of my life where I started a family of my own was a big inflection point on that spectrum. Having a spouse and children is a big commitment of time, and further to the original point: the kids are only going to be around for so long before they go off into the next chapter of their own lives which will involve me less. That limited window increases the importance of getting enough quantity and quality time with them.</p>
<p><img src="https://adamtuttle.codes/img/2020/cruise-family.jpg" alt="A collage of photos of each of my family members with an animal, from a recent vacation" /></p>
<p>Time is all that you have.</p>
<p>You can spend it gaining money, ostensibly to increase the quality of the (now reduced) amount of time that you have with family and friends. You can spend it going to the gym to improve your health and hopefully extend your life. You can learn to solve a variety of shapes and sizes of "twisting puzzles" (rubik's cubes). There are effectively an infinite number of things you could do with it; but in the end, everything comes back to time.</p>
<p>Be deliberate about how you spend it. Take inventory every once in a while and really ask yourself if each interest is worth the time that you're investing into it.</p>
My (First) AWS Fargate Success Story2020-01-24T00:00:00Zhttps://adamtuttle.codes/blog/2020/aws-fargate-success-story/<p>AWS Fargate is amazing! #notsponsored</p>
<p>I started writing this article back in August of 2019, but ended up setting it aside because I didn't know where I wanted to take it. The story that I'm about to tell you ended fantastically for us, and we've since gone on to have even more success with Fargate, so I am happy to sing its praises from the mountaintop.</p>
<p>Back in August we did a thing that was scary and exciting, and it went far better than we ever could have expected. Indeed, far better than developers are conditioned to expect.</p>
<p>We took some code that was designed to run on AWS Beanstalk, and redeployed it on AWS Fargate, and it <em>sorta just worked.</em></p>
<p>Sounds pretty major, right? I figured it would take a week of tweaking knobs and reading overly enterprise-y documentation to find the right incantation to make things work the way we needed. It didn't. We were on our feet in just a few hours. Things went so well that we got scared. <em>What are we forgetting or missing? When will the other shoe drop?</em></p>
<p><a href="https://twitter.com/AdamTuttle/status/1162118705909850112">https://twitter.com/AdamTuttle/status/1162118705909850112</a></p>
<h2 id="our-beanstalk-app%2C-and-why-we-had-to-change-solutions" tabindex="-1">Our Beanstalk app, and why we had to change solutions</h2>
<p>We send a lot of email. More specifically, our customers (colleges and universities) send <em>a lot of email</em>. One part of that process involves flinging api requests at our mail provider as fast as we can (and updating database records to reflect that they've been sent). To reach our desired throughput, we use a small orchestra of relatively long-running (for Lambda, at least) AWS Lambda functions to make the api requests. Each function is in constant contact with a sort of —to continue the orchestra metaphor— "conductor" process that is responsible for keeping tabs on each musician and controlling them to limit our api usage, aiming to get us as close to the rate limit as possible without going over.</p>
<p>The conductor keeps track of how many emails have been sent during a calendar-minute (that is, not a rolling 60-second window, but from 11:27:00.000 to 11:27:59.999) across all running Lambdas, and uses some math on recent throughput and configured limits to decide whether or not to assign a given Lambda thread a new batch of emails to send when it checks in between batches. The Lambda will either be given a new batch, in which case it gets right to work sending them, or it will be told to check back later. The Lambdas themselves keep track of how long they've been running and if they're too close to the timeout deadline they shut themselves down between batches instead of asking for more work. The Conductor also manages the orchestra of running Lambdas. If there's more work that we can fit in during this calendar-minute and we're below our configured max concurrent Lambda workers, it will invoke the Lambda function again, asking it to join the orchestra.</p>
<p>For a particularly large message, we might be sending ~4,500 emails per minute for an hour straight, and use 20-30 Lambda invocations over the course of that hour, but only 3-5 concurrent at any given moment.</p>
<p>Previously, we ran the Conductors on Beanstalk. We had one instance per customer, in part so that we were sure not to let volume from one school affect throughput of another.</p>
<p>It's worth noting that Fargate seems to have been announced <a href="https://aws.amazon.com/blogs/aws/aws-fargate/">in November 2017</a>, and our initial work building the Conductor/Lambda process was done in February of 2018. Not only would we have avoided adopting the service so soon after it launched, but I think that we just didn't know about it. At the time we were a 3 person company, far too busy getting work done to keep up on all of the AWS announcements. In fact, we wrote the majority of the code for this email processing service over one fast paced and sleep deprived weekend. We went with what we were already familiar with -- <em>at least familiar enough</em>.</p>
<p>This system worked well enough from February 2018 until last week, so why change?</p>
<p><strong>We ran out of Elastic IP Addresses. (EIPs)</strong></p>
<p>Beanstalk currently supports two "tiers", one for workers which is intended to consume work items from an SQS queue (not a good fit) and the other for web applications. Though we didn't want any auto-scaling (one of the primary benefits of Beanstalk), it was otherwise a good fit because in additioning to managing the orchestra, we wanted the Conductor to have a web interface where we could watch real-time graphs of throughput and the number of running Lambdas over time, and to allow us to tweak some variables in the process to make sure it was humming along efficiently. We could package up our code as a zip file and deploy it and didn't have to worry about managing the EC2 instances, or wiring up nginx, or a hundred other little things you have to do when you manage your own EC2 instances. This worked really well for us for a long time... But the down side was that every web-tier Beanstalk environment requires you to allocate it one of your Elastic IP addresses. We blocked all external traffic to our Beanstalk apps, so having the EIP was pointless, but it was well worth the $3 EIP rental fee to not have to worry about it.</p>
<p>If you didn't already know, IPv4 addresses are kinda scarce... Not as urgently terrifying as Y2k was in 1999, but scarce enough that each AWS account gets an initial limit of 5. When we hit that limit after a year or so of creating these Beanstalk apps as needed, we figured we'd find a better way later, and asked AWS to grant us a few more IP addresses. Thankfully they obliged. But of course...</p>
<p><a href="https://twitter.com/CodeWisdom/status/1162342494711033856">https://twitter.com/CodeWisdom/status/1162342494711033856</a></p>
<p>We completely forgot about the problem until we ran out of IP addresses again. And of course this time there was no time to waste. It was time to deal with this problem head-on.</p>
<h2 id="fargate-to-the-rescue" tabindex="-1">Fargate to the rescue</h2>
<p>Thankfully my coworker Chad has a personal interest in AWS service offerings and in the meantime had become passingly familiar with Fargate, so we at least had a name for something we should look into. We made a quick list of the things that we absolutely had to be able to get out of our new solution:</p>
<ul>
<li>It can't consume an EIP (or at the very least, we have to be able to turn that off)</li>
<li>It had to be possible to set environment variables that our code could pick up and put to use to configure itself</li>
<li>It had to be able to determine its private IP after launching and create/update an entry in Route 53 so that a hostname could be used for routing messages between the Lambdas and their Conductors.</li>
</ul>
<p>Fargate also happens to work by running your Docker containers, and this is an area we've been wanting to push into more anyway. Win/win!</p>
<p>I quickly made a one-file application that could test all of these requirements, wrapped it up in a docker container, created a registry on AWS Elastic Container Registry, pushed in the test application container, and spun up a Fargate cluster/service/task.</p>
<p>As it turns out, Fargate was able to do all of that and much more.</p>
<p>It took a good week of tinkering before I really understood the distinction between Fargate Clusters, Services, and Tasks and why they are different —probably because this app is one container that doesn't need to cluster, or scale, or connect to any sibling service containers— but with that established, it (Fargate) now feels like it's the sharpest tool in my belt and my instincts now want to use it on every problem. In the meantime, even though I didn't grok it all, we were able to be productive through trial and error.</p>
<p>The hardest part of moving our app turned out to be figuring out how to package it up in a Docker container and publish that container to our registry. We've used Docker before but mostly for local development type purposes, like running a Redis server locally without installing it on your machine. Figuring out the similarities between Docker and Git, and understanding how to build locally and push to a remote registry was the most difficult step, but even that wasn't overly difficult.</p>
<p>Building on some advice from an old friend <a href="https://twitter.com/neurotic">Mark Mandel</a>, we added a <code>Makefile</code> to the project to handle container builds and publishing duties with a couple of simple commands.</p>
<p>With the app successfully wrapped in a container, and the container published to our registry, the only thing left to do was to figure out how to deploy it on Fargate. And that's where the cloud shines! We had a few false-starts where we messed it up and had to trash it and start over, but the only cost was a few cents in compute resources, and our time. Eventually we figured out the right configuration and we had everything we needed.</p>
<p>Maybe next time I'll tell you how we've reinvested those winnings back into Fargate, what sort of awesome improvements we've already made elsewhere in our app, and what sort of awesome improvements are still on our horizon!</p>
<p>I know I sound like a bit of a fan-boy here, but that's only because I am. I am all too excited to have figured out how to use Docker and Fargate to <em>get stuff done</em>, and getting stuff done makes me happy.</p>
Challenge Breeds Stability2020-01-14T00:00:00Zhttps://adamtuttle.codes/blog/2020/challenge-breeds-stability/<p>It's almost a given in the tech industry any more that you have to change jobs every few years. For one thing, getting a new job seems to be the only way to get a substantial pay raise rather than the typical yearly 1-3%. (Disclaimer: I've never worked for a Silicon Valley company so I don't know the first thing about working out there...) And the truth is, especially in the first 5-10 years of your careeer in tech, chances are pretty good that you deserve one or two 15-20% raises because if you've survived that long then you've probably progressed from a micro-managed spec-follower to someone capable of making your own design decisions and at least contributing to the plan.</p>
<p><img src="https://adamtuttle.codes/img/2020/waynes-world-nametags-hair-nets.jpeg" alt="A screen grab of Wayne from the movie Waynes World, smiling awkwardly, next to his extensive collection of name tags and hair nets." /></p>
<p>This march I will have been working for <a href="https://www.alumniq.com/">AlumnIQ</a> for 8 consecutive years. Eight! Not only is that uncommon in general, it's also uncommon based on my personal history. My post-college tech jobs have lasted, in chronological order: 3 years, 3 years, 3 years, and now 8+ years with no end in sight.</p>
<h2 id="what's-different%3F" tabindex="-1">What's different?</h2>
<p>Throughout my career the type of work I've done has been pretty constant: always trending toward more web-dev, and mostly using the same technologies.</p>
<p>As a young dev I learned to love creating admin-areas. I think this was in part because I wasn't any good at web design, so by focusing more on <abbr title="Create, Read, Update, and Delete">CRUD</abbr> forms in an area where the design wasn't as important, I got to feel productive and get stuff done. Eventually I tired of creating CRUD forms over and over and over, though. You can only do so many of them before you have dealt with every possible problem and you're just <em>over it</em>.</p>
<p>My first job was working in IT at a big food company, mostly writing mainframe code to integrate accounts payable and accounts receivable with external systems, but occasionally getting to do some web-dev. After 3 years I left that job to take my second job because it would give me a better opportunity to focus on the tech stack I wanted to be using, and it certainly didn't hurt that it came with a big raise.</p>
<p>My second job was working at a small consultancy (~50 people), on my favorite tech-stack of the time. My manager there opened my eyes to the world of tech blogging and tech conferences. Seeing his articles in trade magazines inspired me to start blogging and attending conferences. With his mentorship, I think this is when I started to grow from some young punk capable of writing CRUD forms into the beginnings of a "senior" developer by some reasonable standard. I started to understand the bigger picture. I was occasionally asked to be interim team lead when the PM was on vacation. I was tasked with many process improvements and initiated things like source control usage (not that our consultancy didn't use any, but some clients didn't), local dev environments, code reviews, and code generators/orm.</p>
<p>(Hey Chuck, thanks for everything!)</p>
<p>After I had been there for about 3 years that company was sold to a larger international consultancy, and slowly but surely our office started to empty out as people were unhappy with the new parent company. The writing was on the wall: the ship was sinking and I needed to find a new home. But by this time I had roots. I got married only a few months before moving to Pennsylvania to take job #2, and by now we had our first kid. Moving wouldn't be as easy any more. So a big part of what influenced my next job selection was local availability.</p>
<p>I ended up finding my next job through Twitter, of all things. One afternoon I met up for drinks with someone I knew casually through tech circles, and eventually the conversation turned to job availability at his company. I ended up getting that job and working there for another 3 years. It was a different job: Still a lot of CRUD forms, but in a different context. We made "simulations and serious games" to let university students experience business concepts for themselves. I got to meet some NFL players and big-shot corporate executives, and working on games made it a novel experience. I decided it was time to go when I could see a tech stack change mandate coming from management, and knew I would hate the new stack.</p>
<p>By this point I had been attending some tech meetups and become friends with Steve. When I decided it was time to leave the university, we started talking about working together. At the time he was solo consulting, but he had big ideas to pivot into products in the university-sports and alumni-engagement verticals. We consulted together for a while to build up some cash reserves and make some strategic customer partnerships, and then jumped into the product world. The sports product ended up not working out and we shut it down, but our engagement products are going gangbusters.</p>
<p>And then all of a sudden, overnight, 8 years went by.</p>
<p><strong>What happened? We found success in products.</strong></p>
<p>We found a pain point that customers were willing to pay to fix, with tremendous room for improvement, in a relatively under-served market. We're still small compared to our competition, but even so, our customers are our biggest champions and we regularly get nice emails letting us know that they saw us hyped in a trade-conference presentation or that our existing customers just talk us up so much that they had to find out if the hype was real.</p>
<p>This kind of success brings its own challenges.</p>
<p>We had done a decent job of following the age-old advice of not prematurely-optimizing, using what we knew, building fast and iterating to find the best path forward.</p>
<p><a href="https://www.flickr.com/photos/69382656@N04/6307867728/"><img src="https://adamtuttle.codes/img/2020/you-must-burn.jpg" style="max-width:100%;" alt="Illustration of a person typing on a keyboard with one hand, while on fire, fighting off a bear with their other hand, an arrow sticking out of their shoulder, missiles visibly inbound, and a car wreck behind them. Caption underneath reads, `You must burn.`" /></a></p>
<p>Now, we've got a product that our customers love, that we have data to prove is making them more profitable and efficient, and that even more people want to buy. In a word, our new problem is <em><strong>scaling</strong></em>.</p>
<h2 id="scaling" tabindex="-1">Scaling</h2>
<p>This is a whole new world. No longer do we get to revel in making the best possible CRUD form. Now we get to deal with challenges like: Sending 1,500 emails per hour isn't fast enough. It's too much work to create an environment to bring new customers online. Deploys are too manual and fragile. And, sadly, our tech stack is a little outdated and that's starting to make things difficult. It served us well in getting here, but it's time to Marie Condo it: Thank it for its service, and then put it in the trash pile, one stained and tattered t-shirt at a time.</p>
<p>So we refactored that email process out of our monolithic app into a <em>sort of microservice</em> utilizing, among other things, some serverless tech, and now we have to self-throttle down to 1,500 emails per minute to prevent DDoS'ing our mail provider.</p>
<p>We're working on containerizing, scripting, and automating deploys.</p>
<p>And we're taking steps to modernize our tech stack. Even with a team of just a couple of developers, we've managed to amass a large product over the last ~6.5 years. You don't just rewrite that all at once and flip a switch. We're being slow and methodical and trying to keep modernization on track while also continuing to expand and improve our product.</p>
<p>It's challenging, and that's a big part of why I no longer get cabin fever even though I've been working from home for 8+ years; and why I am usually excited to get back to work every day.</p>
<h2 id="what's-next%3F" tabindex="-1">What's next?</h2>
<p>It feels like we've <em>ascended</em>. Similar to <a href="https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs">Maslow's Hierarchy of Needs</a>, we've satisfied our ability to survive: we know that our core functionality is useful and resilient, and now we get to focus our energy on some things that make our lives better and enable us to operate at a wider scale with our small, remote team: more-modern tools, developer experience, and operations automations.</p>
<p>I don't know what existence will be like on the plane above this one, but my goal for the year 2030 is to get there and find out.</p>
TIL: Requiring a File that Isn't Exported by a Node Module2020-01-07T00:00:00Zhttps://adamtuttle.codes/blog/2020/til-requiring-node-module-content-not-exported/<p><img src="https://adamtuttle.codes/img/2020/krakenimages-8RXmc8pLX_I-unsplash.jpg" alt="Surprised older man peeks out from behind a wall" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@krakenimages?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">krakenimages</a> on <a href="https://unsplash.com/s/photos/secret?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>Long ago I learned the trick of requiring individual functions from Lodash to keep my bundle size down:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> debounce <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'lodash/debounce'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>I always wondered how it was done (some fancy trick of prototypal inheritance?) but never got around to figuring it out.</p>
<p>Last week I decided that I needed to figure it out, because it would be <em>really</em> handy in my current project with one of our internal company modules.</p>
<p>I should have just trusted my gut and tried it because —spoiler alert— I was right, but first I felt the need to troll around Stack Overflow and Duck Duck Go looking for people who had discussed this technique. I spent easily an hour looking for some sort of guide, digging through the Lodash source on github (not fruitful because their repo is, in their own words, messy right now as they work on a new major version), and when I couldn't find anything helpful I decided to just try some things and see if I could figure it out on my own.</p>
<p>It's really rather easy: You can require any file in a module, even if it's not exported.</p>
<p><code>package.json</code> has an attribute named <code>main</code> which tells node which file (e.g. <code>index.js</code>) to load when you run <code>require('foo')</code>. That file is executed and if it <strong>exports</strong> anything, that export is available to the requiring code as the return value.</p>
<p>But suppose you want a utility baked into the module but that's not exported for... <em>reasons</em>. If that utility is found in the root of your module as <code>util.js</code> then you can use <code>require('foo/util')</code> (or <code>require('foo/util.js')</code> if you prefer). That file <code>util.js</code> may export something, e.g. the utility function we're after. The file you want doesn't have to be in the root of the module. If <code>util.js</code> was in a folder named <code>bar</code> then you could <code>require('foo/bar/util')</code>. It's really just that simple.</p>
<p>Going back to the Lodash example, the reason you can <code>require('lodash/debounce')</code> or <code>require('lodash/partition')</code> is because each of those functions is available as a file in the root of the module.</p>
<p>Hopefully this helps get some information on this practice into search engines, and hopefully I can find it the next time I'm wondering how it's done.</p>
My SMART Goals for 20202019-12-30T00:00:00Zhttps://adamtuttle.codes/blog/2019/my-smart-goals-for-2020/<p>I don't make resolutions, I make goals.</p>
<p>A popular approach to setting goals is to <strong>make them SMART — that is: Specific, Measurable, Attainable, Realistic, and Timely.</strong> I don't recall where I first heard of this acronym, but what really made it stick for me was becoming a skydiving coach. We're taught to give our students SMART goals because (let's use the short version here)... they work!</p>
<h2 id="what-makes-goals-smart%3F" tabindex="-1">What makes goals SMART?</h2>
<p>I'll use my first goal to illustrate: I want to reduce my body fat percentage (BF%) from 21% to 15% and increase my strength by an appreciable amount by the end of 2020.</p>
<p><img src="https://adamtuttle.codes/img/2019/eoy-bodyfat.png" alt="A chart showing my weight and body fat percentage over time, slowly trending down" /></p>
<p>I don't want to "lose weight," I want to lose a certain amount of weight. And actually, part of this goal is to increase muscle mass, and muscle weighs more than fat. Really, I want to lose BF% not weight. If my body fat goes down but my weight goes up, that would actually make me super happy because it means I put on a bunch of muscle mass! I own a scale that uses electrical resistance to tell me my body fat percentage in addition to my weight, so that takes care of specific and measurable.</p>
<p>I'm not setting a specific strength goal (e.g. bench press 200 lbs) because my goal isn't to get yoked, it's to know that I'm taking care of myself because I can empirically say I'm getting stronger.</p>
<p>Let's get serious here: What I most want is my 20-year-old physique back! I want those lines where my abs meet my hips. Is that attainable and realistic in a year? Definitely not.</p>
<p>So what's attainable? A reduction of a few % points for the year sounds reasonable. I'm about 168.4 lbs and 21% BF right now, which means I'm carrying 35.4 lbs of fat around (<em>sounds gross when you put it that way!</em>). Assuming I maintained a weight of 168 lbs, 15% BF would be 25.2 lbs of fat. That would mean I'd replaced almost 10 lbs of fat with muscle. It sounds daunting, but we're talking about an entire year here. I'm going to call that attainable and realistic for me.</p>
<p>Timely is the easy part: Hit it by midnight on 12/31/2020. I'm also going to set milestone goals for every other month.</p>
<p>To jumpstart the process, I've been eating Keto (with the enthusiastic support of my doctor!) and following the workout advice of online personal training service <a href="https://devlifts.io/">DevLifts</a>, since November 1st. I started at 25% body fat and 182.1 lbs and I'm down to 21% and 168.4 lbs already. I'm sure that includes the low hanging fruit like water weight, so that's why my BF% goal for 2020 isn't as aggressive. Thanks in part to a few cheat meals and missed gym days around the holidays, I seem to have plateaued a little bit, but I'm confident that I can get back on the wagon.</p>
<p>To summarize: goal #1 is to reduce my body fat percentage from 21% to 15% and increase my strength (by logging all weights and reps and seeing an appreciable improvement) by the end of 2020.</p>
<h2 id="what-am-i-doing-to-help-myself-be-successful%3F" tabindex="-1">What am I doing to help myself be successful?</h2>
<p>Here's a tip I've found really useful for my personality: Habit tracking apps. But you have to find the right one for you. I've tried a few over the years that gamified accomplishing your goals (like Habit RPG), but found that the juice wasn't worth the squeeze and I eventually lost interest and my progress fell off a cliff.</p>
<p>So gamification isn't really for me, but you know what is? If it wasn't obvious from the chart above... Data! I'm a total data nerd.</p>
<p><a href="https://adamtuttle.codes/img/2019/habits-1.png"><img src="https://adamtuttle.codes/img/2019/habits-1.png" alt="screen shot of my phone showing " style="max-width: 24%" /></a> <a href="https://adamtuttle.codes/img/2019/habits-2.png"><img src="https://adamtuttle.codes/img/2019/habits-2.png" style="max-width: 24%" /></a> <a href="https://adamtuttle.codes/img/2019/habits-3.png"><img src="https://adamtuttle.codes/img/2019/habits-3.png" style="max-width: 24%" /></a> <a href="https://adamtuttle.codes/img/2019/habits-4.png"><img src="https://adamtuttle.codes/img/2019/habits-4.png" style="max-width: 24%" /></a></p>
<p>This is the Android app (Sorry iPhone people!) <a href="https://play.google.com/store/apps/details?id=org.isoron.uhabits">Loop Habit Tracker</a>, which I heard about on the podcast <a href="https://syntax.fm/">Syntax</a>.</p>
<p>Being able to see my streak right there as a widget on the home screen of my phone, and getting push notifications reminding me to mark off the days that I keep the streak alive is super motivating.</p>
<p>Obviously I've really let myself go crazy on the soda in November and December. Not only did I completely fail to avoid soda even once this month, I've allowed myself to creep back up to multiple per day. Time to put some effort into getting back on that wagon.</p>
<p>I managed not to have any on Sunday, the day that I wrote this. If I can do it again tomorrow, that's two in a row. And if I do it again the day after that? <a href="https://www.youtube.com/watch?v=MLCLMEYp9s0">That's called a winning streak</a>.</p>
<h2 id="what-are-the-rest-of-my-smart-goals-for-2020%3F" tabindex="-1">What are the rest of my SMART goals for 2020?</h2>
<ol>
<li>Reduce my body fat percentage from 21% to 15% and increase my strength (by logging all weights and reps and seeing an appreciable improvement) by the end of 2020.
<ul>
<li>Keep my record of hitting my macros above 90%. That's 36 unsuccessful days in the year. Perhaps a bit ambitious... but reach for the stars, right?</li>
<li>Work out 6 days per week and never miss more than 2 workouts per month for any reason (92% success rate)</li>
<li>It's no coincidence that I want to reduce my body fat by 6 percentage points... That's one point every 2 months. Those are my milestone goals.</li>
</ul>
</li>
<li>Build the super secret Christmas present I already have planned for my wife for Christmas 2020! It will be the single largest and most expensive build I've ever attempted. Alas, I can't share any more details than that here because there's a small chance she might read this.</li>
<li>Build my CNC and learn how to use it... By using it for birthdays and other upcoming gifting opportunities this year.</li>
<li>Wean back down to (nearly) no soda, and then keep the success rate over 50% on average for the year.</li>
</ol>
<p>What are your goals for 2020, and how are you making them SMART?</p>
The Config Problem2019-12-18T00:00:00Zhttps://adamtuttle.codes/blog/2019/the-config-problem/<p>Architecture discussion time!</p>
<p>Our primary product started its life as a monolith, and as features started to outgrow the capabilities of our monolithic stack we fractured them off as microservices. To my mind, this sounds like two gold-plated best practices exercised well:</p>
<ol>
<li>Make it work, then make it good, then make it fast.</li>
<li>Avoid premature optimization.</li>
</ol>
<p>In an attempt to be pragmatic, the original solution I came up with for managing configuration was to hard-code it all into a data structure in a class. (<em>"We can improve it later when it becomes a problem!" Sometimes I want to punch prior me in the mouth...</em>) Since we were writing CFML it became known as <code>config.cfc</code>. Using a framework with great support for Dependency Injection made it easy to pull that config class into any controller or service throughout the app.</p>
<p>It's not super important to the story but it provides context for what comes next: We host that monolithic app on an AWS EC2 instance.</p>
<p>When certain scheduled tasks started to bring the rest of the application to its knees, the first thing we did was to bring up a 2nd server to run background tasks. That way if a scheduled task brought everything grinding to a halt (again), users wouldn't be affected. No config problem here. There's two copies of <code>config.cfc</code>, but they're both checked out of the same repository and unchanged, so there's nothing to worry about.</p>
<p>Then things got hectic. Some features got good enough that they became mission critical to our customers, and their demands for speed and throughput were simply more than our stack could handle, even with a server dedicated to running background tasks.</p>
<h2 id="to-the-cloud!" tabindex="-1">To the cloud!</h2>
<p><img src="https://adamtuttle.codes/img/2019/to-the-cloud.jpg" alt="To the cloud!" /></p>
<p>Since we knew we had created a valuable tool but it couldn't keep up with demand, we knew it was time to optimize its performance. We did that thing you're not supposed to do: We gathered the entire team in one physical location, locked ourselves in a room, and coded and talked about code until our eyes and our ears and our brains hurt, slept a few hours, and then got right back to it. Repeat until success or failure. I wish I could tell you that in that moment we somehow magically had the type of work-life balance that everyone dreams of and still managed to solve our problems fast enough.</p>
<p>Nope.</p>
<p>We did the unhealthy thing. The "hero" thing. And I can't recommend it.</p>
<p>But it worked.</p>
<p>When we came out of that weekend we had a set of microservices that were capable of being several orders of magnitude faster than what the old monolith could pull off. In fact it was so fast that we had to add artificial limits to prevent the jobs from choking the database to death or exceeding an entire hour's api rate limit in the first 5 minutes; and even with those artificial limits the new processes were still 2+ orders of magnitude faster.</p>
<p>But herein lies the rub.</p>
<p>The new processes run on AWS Lambda, running Node.js. Node can't (or at the very least, shouldn't!) handle cfc files, so what we did at that time was to duplicate our configuration and create a node module that mostly works the same way that <code>config.cfc</code> does. You require our node module and get back some functions that allow you to pull out any setting you want.</p>
<h2 id="problem-%231" tabindex="-1">Problem #1</h2>
<p>Keeping these two parallel configs in lock-step has not been easy; but it's been several years and we've instilled a decent routine that keeps us mostly on top of this problem.</p>
<p>Ideally we would have only one source of configuration-truth, and all applications could consume it.</p>
<p>The natural choice here would seem to be JSON, not least because the native data structures we've used to this point in the CFC and the node module heavily resemble JSON. If we could figure out the logistics, any app we've got could easily parse a JSON file at startup to load the config into memory.</p>
<p>Ah, but the logistics. Let's start with the simple solution and see where it falls over.</p>
<p>Why not add a <code>package.json</code> and npm-install our node config module into our CFML app, reading it at app startup?</p>
<p>That much is fine, but now every time we need to make a configuratoin change we have to follow all of these steps in the right order:</p>
<ul>
<li>Make the config change in the node config module repo</li>
<li>Tag and push a new release of the config (PR with code review and approval process)</li>
<li>Update the dependency in our CFML app's package.json to require the new config module version</li>
<li>Tag and push a new release of the app (PR with code review and approval process)</li>
<li>Deploy the app changes, including manually updating dependencies after app deploy</li>
</ul>
<p>Yes, it's true, we don't have any deploy automation yet. It's on our short list of projects to get to sooner rather than later and the groundwork is already in progress, but for the moment we still deploy manually.</p>
<p><em>Yes, it does feel like the web development equivalent of banging rocks together. Thanks for asking.</em></p>
<p>And all of that only covers the monolithic core app. What about our microservices?</p>
<h2 id="problem-%232" tabindex="-1">Problem #2</h2>
<p>Our microservices are mostly Lambda functions, which are a snapshot in time. Just deploying a new version of the config doesn't mean the Lambda functions will start using it. We currently have to:</p>
<ul>
<li>Update the dependencies in our Lambda functions' package.json files to require the new config module version</li>
<li>Tag and deploy a new release of the service (PR with code review and approval process)</li>
<li>Repeat for each Lambda function</li>
</ul>
<p>Fortunately we use the AWS CLI to deploy Lambda updates, so those get done quickly, even if still a bit tedious.</p>
<p>We're not using private npm modules. Instead we're using private git repos and npm's ability to load a git repo as a dependency using a git url. There's even <a href="https://semver.org/">semver</a> support:</p>
<pre class="language-json"><code class="language-json"><span class="token punctuation">{</span><br /> <span class="token property">"dependencies"</span><span class="token operator">:</span> <span class="token punctuation">{</span><br /> <span class="token property">"config"</span><span class="token operator">:</span> <span class="token string">"git+https://<secret-key>:x-oauth-basic@github.com/alumniq/config.git#semver:^3.0.0"</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>I've never worked in a monorepo before, but I get the feeling that this is the type of problem that the monorepo design pattern is built to solve.</p>
<h2 id="a-stab-in-the-dark" tabindex="-1">A stab in the dark</h2>
<p>Writing all of this up has made me wonder if AWS offers a configuration service of any kind. They probably do. Kinda have to, right? Something we've discussed internally is the idea of putting our config behind an API that's only accessible inside our VPC. I'm guessing a config service product would work similarly.</p>
<p>I also think that our current strategy of updating <code>package.json</code> for each individual config version may be a symptom of a problem that's already been fixed. I believe there used to be a bug in npm that only affected git-url dependencies and would not install anything but the exact version number listed, even if you used <code>^</code> to indicate that minor- and patch-increases are acceptable. We burned many hours hunting that down (several times in different places) and now we have muscle memory to keep that version number up to date to prevent problems -- but I think the bug might be fixed and we could probably get away with a simple <code>npm update</code> to update <code>package-lock.json</code>, and a deploy.</p>
<p>There's probably a few of you out there screaming at your screen because I'm <em>obviously an idiot</em> and should be using ... some product or service... that solves these problems for me. And to you I say, please make use of the comment form! I'm eager to improve this situation.</p>
Slow Is Smooth, Smooth Is Fast2019-12-10T00:00:00Zhttps://adamtuttle.codes/blog/2019/slow-is-smooth-smooth-is-fast/<p>In skydiving we have a saying: Slow is smooth, and smooth is fast.</p>
<p>When you get out of an airplane at 14,000 feet in the air, you have somewhere between 40 and 60 seconds to play before you need to do the important work of saving your life by opening your parachute. (The timing depends on what exactly you're doing while you fall.)</p>
<p>Once you get beyond the concept that it's possible to get out of an airplane in flight and then touch down softly, you start to want to hang out with friends while you're there.</p>
<p><img src="https://adamtuttle.codes/img/2019/sit-fly-winter-2019.png" alt="Me, skydiving" /></p>
<p>This has actually turned into a sport with a lot of different disciplines within it. For example, here's the 2019 performance of this year's national champion team, Rythm:</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/etTZSNgIm-A" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
<p>Like most other sports, the goal here is to get the most "points." In formation skydiving a point is a completed formation, possibly following a compulsory movement. Basically, turn a certain way and be holding onto each other in the right spots.</p>
<p>As you can imagine, it's easy to feel pressured to move quickly. You have limited time and the goal is to get as many points as you can, and hope you get more than the other teams. But somewhat counterintuitively, most professional coaches will teach their teams to slow down and be deliberate about their movements. They say, "Slow is smooth, and smooth is fast."</p>
<p>The meaning is that when you're taking enough time to be deliberate and calculated in your movements, you're less likely to make mistakes. And the fewer mistakes you make, the faster you can go. If you instead act frantically, you're more likely to make mistakes and those mistakes are more likely to cost you both in the moment and again later.</p>
<p>In skydiving, that might mean you miss your grip now and you also put yourself in a harder to reach spot in the sky for a future formation, slowing the team down even more.</p>
<hr />
<p>I was reminded of this saying yesterday when, in an attempt to multitask and move (with the clear eyes of hindsight) what was admittedly a little bit frantically, I accidentally deleted the most important partitions on my computer: The operating system I use every day at work.</p>
<p>You may know that <a href="https://adamtuttle.codes/blog/2019/building-a-hackintosh-2019/">I use a Hackintosh that I built for myself this year</a>. I built it on some pretty great hardware, and I'm attempting to set it up to dual-boot Windows for more options in PC gaming. Thinking I was <em>sooo clever</em> I setup the installer to run while I would be showering. Multitasking for the win! Except, after my shower as I was brushing my teeth, I realized what I had done. (Basically: I forgot to unplug the OSX drive and plug in the empty drive for Windows to install onto.)</p>
<p>This story is not a complete tragedy, because thankfully I had a recent complete drive clone available to recover with, but man did it feel like a gut-punch at the time! Trying to rush really bit me, and if I wasn't prepared that could have ruined a lot more than my day.</p>
<hr />
<p>The same principle is true for engineering work. Taking the time to be deliberate, you'll make fewer mistakes and have cleaner solutions. It might mean you'll have more, better, (any?) automated tests, which not only give you confidence in what you're bulding now, but also provide confidence that you haven't inadvertently broken it later.</p>
<p>Too often in the software world we're focused on the deadline, and then what happens? I bet you can relate to this: You put in extra hours and stress yourself out, but you meet that deadline. Then the customer ends up not touching your work for months. How infuriating!</p>
<p>I'm not here to tell you how to estimate projects and how to account for the time to write tests and documentation (if you've got that magic mirror, do share!) but what I am saying is that you can probably afford to take the time to be deliberate. If you do, it will save you time in the long run.</p>
Notes on Notes Apps2019-11-26T00:00:00Zhttps://adamtuttle.codes/blog/2019/notes-on-notes-apps/<p><img src="https://adamtuttle.codes/img/2019/nathalia-rosa-P1SdQAhcJz8-unsplash.jpg" alt="Lots and lots of sticky notes stuck to a wall" /></p>
<p>I'd love to have a conversation about note taking, and especially about note taking apps.</p>
<p>What do you use, how do you use it, and what makes it work well for you? I realize these are big questions and probably worthy of linking out to your own blog post, if you have a blog. Please do! Or leave a comment, or link to a youtube video you like, or whatever works. I just need a better solution for taking notes.</p>
<p>I've tried dozens of note taking apps over the years and never found one that works well for me.</p>
<p>At the moment I'm using Google Keep, because it's easily shared between my phone and computer, and it's... ok. I don't have a ton of information in it, but I can already see how unweildy it will get once I do have lots of notes. Is there a good way to keep it organized?</p>
<p>I tried Notion, but I think it's a little too open-ended for me. I found it overwhelming to figure out how to use it in the first place, and then since it's so customizable, every little job takes a bunch of effort to get it to look and act how you want. I'd rather fit someone else's mold as long as it's close to what I'd like and does most of the work for me.</p>
<p>I've heard good things about <code>gatsby-theme-notes</code> but I haven't tried it yet. From what I've seen it mostly seems like some Markdown files with decent formatting and you can put them in folders?</p>
<p>So I'm dying to know. Where do you keep notes? What do you like about it? What do you hate about it? How do you keep them well organized and stay productive?</p>
Is Imposter Syndrome Actually Good for You?2019-10-15T00:00:00Zhttps://adamtuttle.codes/blog/2019/is-imposter-syndrome-actually-good-for-you/<p><img src="https://adamtuttle.codes/img/2019/two-kids-in-a-trench-coat.jpg" alt="Two kids in a trech coat" /></p>
<p>Given my blog-pivot to my evolution into management, it feels right to open things up by shining a light on the concept of starting new things. I want to look at how I felt on my first day as a newly minted CTO. The elephant in the room is <strong>Imposter Syndrome</strong>.</p>
<p>You can downplay it, if you want. <em>Sure, I'm the CTO but I'm also literally half of the company. There's nobody under me. Yet.</em></p>
<p>But consider this: I've yet to see a scenario that you could plausibly find yourself in where your imposter syndrome might actually turn out to be true.</p>
<ul>
<li><strong>New job?</strong> All of the people you interviewed with seem to think you're qualified!</li>
<li><strong>First speaking engagement?</strong> You were chosen over probably hundreds of others.</li>
<li><strong>Failed to reach a personal goal?</strong> You're ambitious and set challenging goals. No shame there.</li>
</ul>
<p>You have real value, and your imposter syndrome is anxiety for a future that will probably never come. If you push through that anxiety and do the best you can, you'll probably find that you're doing a great job.</p>
<p>I'm starting to wonder if imposter syndrome is nothing more than a sign that you're pushing yourself to grow. A personal barometer for finding the edge of your comfort zone. If you frame it that way, it becomes a positive thing. A useful tool on your belt.</p>
<p>Imposter syndrome is real, and the way it makes you feel can really affect you. But just like stubbing a toe in the dark, if you choose to accept it and move on with your life, you'll still get to where you were going.</p>
Find a Way to Get Work Done, or Make One.2019-09-30T00:00:00Zhttps://adamtuttle.codes/blog/2019/find-a-way-to-get-work-done-or-make-one/<p><img src="https://adamtuttle.codes/img/2019/noel-o-shaughnessy-DAOXI14Irx8-unsplash.jpg" alt="A leaf-covered path through the woods" /></p>
<p>The title of this post is a callback to a blog post I read seven and a half years ago. I really wish I still had or could find the link, because I would love to link it. It was based on a quote attributed to Hannibal, which went something like, "We will find a way, or we will make one." I've probably butchered it, because it was more than seven years ago, and because I can't find it. I'm sorry. Please <a href="https://twitter.com/adamtuttle">hit me up</a> if you can find it!</p>
<p>It was a blog post and open letter from a startup founder to his early-stage employees; and it was about empowering them to do what they needed to do to accomplish their mission. They all had the company credit card and the authority to use it (responsibly) to push them toward success. Hence, "Find a way, or make one."</p>
<p>It was my then-soon-to-be business partner Steve that sent me the link. I got it in an email from him on my way to a conference. He sent it while I was on my flight, and I remember sitting in a packed SuperShuttle on a hot day next to a couple of sweaty dudes, sun blaring in the window, rapt, reading it on the way to the conference hotel. <em>It made that kind of impression on me.</em> We had recently decided that I would quit my corporate job and come work with him, and that we wanted to create a product that would eventually become <a href="https://www.alumniq.com/">AlumnIQ</a>.</p>
<hr />
<p>It's been seven years and change since we started that adventure. AlumnIQ now has five full time employees, 10 customers for our primary product and dozens for our <s>gateway drug</s> secondary and tertiary products. We have <em>a lot</em> of code and, it probably won't shock you to learn, a lot of technical debt too.</p>
<p>All along the way we've <a href="http://boringtechnology.club/">chosen boring technology</a> to allow us to spend our mental energy on solving hard problems. We intentionally avoided premature optimizations and built things with tech and strategies that we knew would eventually need to be replaced, because what we didn't know was how soon they would need to be replaced. This turned out to be just what we needed. We amassed a healthy pile of technical debt in service of getting our product into the hands of our customers and getting profitable.</p>
<p>And then we became successful. More successful than we were ready for, faster than we expected, and before we knew it was happening that healthy technical debt we intentionally accepted early on took an unplanned turn toward unhealthy. Things started to feel more like a maxed-out credit card where most of your monthly payment goes toward interest.</p>
<p>Here's where we made a mistake: Like every dev shop I've ever worked for in the past, we don't have nearly enough automated tests. <strong>(Wait... Am I the problem?!)</strong> And if my experiences, and discussions at conferences are indicative of anything, that's also true for most businesses. We do test things, but most of that is done manually. You know what manual testing is terrible for? Finding regressions. When fixing one thing breaks something else.</p>
<p>The frustrations we're experiencing with 5 employees and 10 customers are only going to get worse as we continue grow. And for... <em>reasons</em>... I am certain that those frustrations are not going to grow linearly. Probably more like exponentially. We need to get our testing situation improved, and fast.</p>
<p>Fortunately, automated testing is a pretty well understood problem space, right?!</p>
<p>Well, one of those <em>boring tech choices</em> we made was to use the coding language CFML, because Steve and I were both well versed in it and able to be productive on day 1. And CFML is <em>fine</em> (not a discussion I'm interested in right now!) but it certainly doesn't have much mind-share in 2019. There aren't many people working on (open source projects for the good of the community in) the frontier of good ideas, and a lot of things get left to wither on the vine a bit while their creators are off carrying the torch in other interesting problem spaces.</p>
<p>Case in point: Continuous integration tech for CFML is possible, but frustrating to figure out and options are limited. How do I know? I just spent two days sorting through it all.</p>
<p><a href="https://twitter.com/AdamTuttle/status/1178068571005558786">https://twitter.com/AdamTuttle/status/1178068571005558786</a></p>
<p>As I mentioned in my tweet, I've figured it out. But it wasn't easy and it ultimately only worked after a dozen failed attempts, and only once I put together several puzzle pieces of more recent tech in the CFML world. So with huge thanks to the community around TestBox and CommandBox, I'm happy to share my proof of concept repo for running CFML continuous integration tests on Travis-CI.</p>
<p>Here's the repo: <a href="https://github.com/AlumnIQ/CFML-CI-2019">CFML-CI-2019</a>. It's based heavily on Matt Gifford's similar repo <a href="https://github.com/coldfumonkeh/cfml-ci">cfml-ci</a>, but here are the problems that I ran into and how I solved them:</p>
<ul>
<li>Matt's repo was last updated in November of 2016. Circle CI has released a v2 of their service, and it doesn't seem like they still support their v1 configs, which is what Matt was using. V2 is based heavily on Docker which, especially when used in combination with CFML engines, is not my strong suit. I ripped out all pretense of Circle CI support/examples. While I was at it, I also ripped out MXUnit support because (1) it's 2019, and (2) I didn't have any MXUnit tests to run. It would be trivial to add that back in if you need it, though. (See cfml-ci.)</li>
<li>Matt's Travis CI support was built around Ant and manually installing the CFML server. Since cfml-ci hasn't seen updates recently, the CFML server versions that are important to me aren't already included. I started looking around for other CFML projects that I knew used Travis and remembered that FW/1 does, so I peeped <a href="https://github.com/framework-one/fw1/blob/1c946217c1aae1a58e8569b366905f8b08d68c46/.travis.yml">their config</a>. I was excited to see that it uses CommandBox both for installing the server and for running the tests. Fortunately this is a space I had been wading into over the last few weeks anyway, so it was a happy coincidence. Ultimately this realization is what put me on the path to success.</li>
<li>Lastly, I found <a href="https://testbox.ortusbooks.com/continuous-integration/travis">this page in the TestBox docs</a> describing how to get running on Travis. Cribbing from those various sources, as well as troubleshooting on the CFML Slack (a fantastic resource!), I was able to get my proof of concept running.</li>
</ul>
<p>I've been able to take this proof of concept and adapt it to our main product repo, and now we're getting test results right in our pull requests!</p>
<p><img src="https://adamtuttle.codes/img/2019/automated-testing-on-travis.png" alt="Screen shot of GitHub pull request merge dialog showing that tests have been run in the cloud and passed for this PR!" /></p>
<p>I'm not promising to keep this repo up to date as Travis, TestBox, and CommandBox evolve, but it should at a minimum serve as a useful snapshot of what's required to run CFML TestBox tests on Travis in late 2019. I'd be willing to port my changes into cfml-ci, and I have reached out (<a href="https://github.com/coldfumonkeh/cfml-ci/issues/10#issuecomment-536204823">sort of</a>) to see if there's interest in that... Likewise, if anyone wants to contribute to my repo, that's welcome too.</p>
<p>What does this have to do with "Finding a way, or making one"? Our tech debt has been causing burnout around the team, and making it recur much more rapidly, too. Some days I would struggle to overcome my sense of dread at how bad things can get when you don't stop to take stock often enough. Solving this testing problem was my way of pushing through the pain and making a way to enjoy my work again.</p>
<p>I realized that we have an elephant in front of us and a mandate to eat it. And there's only one way to do that: One bite at a time.</p>
<p>Make one small improvement that measurably moves you closer to your goal. Then figure out what comes next, and make one more small improvement. Repeat that enough times, and pretty soon the elephant disappears.</p>
<p>That's the theory, anyway.</p>
Moving From Adobe ColdFusion 11 to Lucee, a Case Study2019-09-18T00:00:00Zhttps://adamtuttle.codes/blog/2019/from-coldfusion-to-lucee/<p><img src="https://adamtuttle.codes/img/2019/erda-estremera-sxNt9g77PE0-unsplash.jpg" alt="Moving Day: A small dog in a moving box" /></p>
<p>My team has made the decision to migrate our application from Adobe ColdFusion 11 ("ACF") to Lucee 5, and the time has come to pull the trigger. I spent a couple of days this week trying to get the app on its feet running on Lucee, and now that I seem to have solved all of the initial challenges, I thought I'd write about what problems I had and how I resolved them while it's still mostly fresh in my mind.</p>
<p>Plus I still have my git commits all neatly organized in a branch for easy reference.</p>
<p>I'm sure that this won't be an exhaustive list of every possible difference that you might encounter. Our app doesn't use every feature of the platform. But here's an account of what we had to deal with.</p>
<h2 id="orm" tabindex="-1">ORM</h2>
<p>By far, our biggest problem stemmed from how heavily this application uses the Hibernate ORM features in ACF. We have something like 160 distinct entities, with a lot of relationships between them.</p>
<h3 id="mappedsuperclass" tabindex="-1">mappedSuperClass</h3>
<p>To make matters worse, we <em>"architected"</em> ourselves into a corner early on in the project by (sigh, we thought we were clever...) creating a parent object for every entity and using <code>mappedSuperClass</code> and inheritence to allow ourselves to customize certain entities for certain customers in certain scenarios. (ugh.)</p>
<p>I never ran into any problems that seemed to be caused by our usage of <code>mappedSuperClass</code>, but then again, ripping that out was one of the first things that I did. I got rid of all of the parent objects and inheritence. I guess it's fair to note that I've gotten mixed reports on <code>mappedSuperClass</code>. Some claim that it won't work with Lucee, and others say it's worked fine for them. It doesn't affect us, because we've wanted to get rid of it for a long time anyway, and now it's gone.</p>
<p>Similarly, I got mixed reports about using <code>lazy=true</code>/<code>lazy='extra'</code> on relationships. I left our lazy relationships in place and they seem to be working... for now.</p>
<h3 id="error-messages" tabindex="-1">Error messages</h3>
<p>The biggest problem debugging ORM issues is the error messages. They never seem to offer any useful hints; or they're at least <em>way</em> too subtle.</p>
<p>Initially, after I had ripped out our usage of <code>mappedSuperClass</code> and deleted the parent objects, none of my entities had <code>persistent=true</code> set, so when the application attempted to load a user bean to put into the session, it reported this error:</p>
<blockquote>
<p>No entity (persistent component) with name [User] found, available entities are []</p>
</blockquote>
<p>With the benefit of hindsight, the "persistent component" parenthetical does seem to be smacking me in the forehead, but at the time it was pretty baffling why it was saying that we didn't have any entities. We had many! Sort of.</p>
<hr />
<p><img src="https://adamtuttle.codes/img/2019/lucee-orm-error-sample.png" alt="Entity Name [EventProductFee_PersonType] is ambiguous, [/Users/atcodes/DEV/AlumnIQ/com/alumniq/orm/ems/EventProductFee_PersonType.cfc] and [/Users/atcodes/DEV/AlumnIQ/com/alumniq/orm/ems/EventProductFee_PersonType.cfc] use the same entity name." /></p>
<p><em>DUH</em>, right?</p>
<p>This one turned out to be that in Application.cfc, in <code>ormSettings.cfclocation</code> we had both the <code>orm/</code> folder, and the <code>orm/ems/</code> folders listed. While ACF managed to recognize that the same files were listed twice (because it was more than happy to recurse from <code>orm/</code>) and dedupe them, Lucee recursed but doesn't seem to be interested in deduplicating them and instead throws this error.</p>
<hr />
<p>Next up was this lovely gem:</p>
<blockquote>
<p>entity [Person] with cfc name [orm.ems.Person] does not exist, existing entities are [foo,...]</p>
</blockquote>
<p>This one turned out to be caused by relationship definitions. Some other entity had a relationship to Person and defined it as <code>cfc="orm.ems.Person"</code>, but Lucee didn't like that. For some reason it made me drop the <code>orm.</code> prefix.</p>
<h3 id="strict-in-places-acf-is-not" tabindex="-1">Strict in places ACF is not</h3>
<p>Another early foible in our journey into the CF-ORM rabbit hole was cleverly thinking that we would group tables in the database by prefixing their table names, but not prefix the entity file name, as in:</p>
<p><strong>orm/ems/Activity.cfc:</strong></p>
<pre><code>component
output="false"
persistent="true"
table="EMSActivity"
{
...
}
</code></pre>
<p>Of course, ACF doesn't seem to care what's in the <code>table</code> attribute. It named our tables after the CFC file name, so in this case we got "Activity". At the time we ignored it. So our tables aren't grouped. Who cares?</p>
<p>It became a problem when switching to Lucee. I had to find all of the entities that attempted this approach and remove the prefix so that it would match the existing table.</p>
<h2 id="different-approaches-to-java-classes" tabindex="-1">Different approaches to Java classes</h2>
<p>Lucee 5 <a href="https://luceeserver.atlassian.net/browse/LDEV-2317">doesn't currently support <code>this.javaSettings</code></a> in Application.cfc, instead favoring an <a href="https://docs.lucee.org/guides/lucee-5/osgi.html">OSGi approach</a>.</p>
<p>So while this approach worked fine on ACF, for loading JARs for things like BCrypt and Redis:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">this</span><span class="token punctuation">.</span>javasettings <span class="token operator">=</span> <span class="token punctuation">{</span><br /> loadPaths <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token string">"../lib"</span><span class="token punctuation">]</span><span class="token punctuation">,</span><br /> loadColdFusionClassPath <span class="token operator">=</span> <span class="token boolean">true</span><span class="token punctuation">,</span><br /> reloadOnChange <span class="token operator">=</span> <span class="token boolean">false</span><span class="token punctuation">,</span><br /> watchInterval <span class="token operator">=</span> <span class="token number">60</span><span class="token punctuation">,</span><br /> watchExtensions <span class="token operator">=</span> <span class="token string">"jar,class"</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre>
<p>Those JARs and CLASSes aren't loaded by default in Lucee. Instead, you have to add a third argument to your <code>CreateObject()</code> calls:</p>
<pre class="language-js"><code class="language-js">pool <span class="token operator">=</span> <span class="token function">CreateObject</span><span class="token punctuation">(</span><span class="token string">'java'</span><span class="token punctuation">,</span> <span class="token string">'redis.clients.jedis.JedisPool'</span><span class="token punctuation">,</span> <span class="token function">expandPath</span><span class="token punctuation">(</span><span class="token string">'/lib'</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>This wasn't too cumbersome to overcome. I searched (using regex) for <code>createObject\(\s?["']java</code>, which found all createObject calls that load java objects. Ignoring any that used baked-in system classes (like <code>java.lang.Thread</code>), and adding the 3rd argument for any that used libraries we're adding.</p>
<h2 id="ehcache-isn't-running-by-default" tabindex="-1">EhCache isn't running by default</h2>
<p>Using the methods <code>cacheGet</code>, <code>cachePut()</code> etc all depend on EhCache in ACF. In Lucee caches are not configured/running until you create them. I had to create a cache (choosing between RAM and EhCache) in admin, and then once it was created I could set it as the store for object caches.</p>
<h2 id="syntax-differences" tabindex="-1">Syntax differences</h2>
<h3 id="no-support-for-for-in-loops" tabindex="-1">No support for for-in loops</h3>
<p>For-in on arrays is handy. Oddly it seems like we only had one of these. I changed it from:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">var</span> deletes <span class="token operator">=</span> <span class="token string">''</span><span class="token punctuation">;</span><br /><span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> tbl <span class="token keyword">in</span> tables<span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> deletes <span class="token operator">&=</span> <span class="token string">'drop table #tbl#; '</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>to:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">var</span> deletes <span class="token operator">=</span> <span class="token string">''</span><span class="token punctuation">;</span><br />tables<span class="token punctuation">.</span><span class="token function">each</span><span class="token punctuation">(</span><span class="token keyword">function</span> <span class="token punctuation">(</span><span class="token parameter">tbl</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> deletes <span class="token operator">&=</span> <span class="token string">'drop table #tbl#; '</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p><strong>Update:</strong> Lucee <em>does</em> support for-in loops on arrays. I'm not sure why that particular loop I ran into seemed to be troublesome, but it certainly was at the time. Probably. I think. But as you can see in the example below, it definitely works. 🤷🏻♂️</p>
<iframe style="width: 100%; height: 250px;" src="https://trycf.com/gist/da195986b3333a006376aa2165b1b203/lucee5?theme=monokai"></iframe>
<h3 id="canonicalize" tabindex="-1">Canonicalize</h3>
<p>It's <a href="https://cfdocs.org/canonicalize">reasonably well documented</a> that ACF's final argument to <code>canonicalize()</code>, <code>throwOnError</code> <a href="https://luceeserver.atlassian.net/browse/LDEV-2391">isn't supported by Lucee</a>. It's still an open question for me to figure out how this affects our app (because we had it set to true), but for now I've simply deleted the argument.</p>
<h3 id="colons-in-function-meta-attribute-names" tabindex="-1">Colons in function meta-attribute names</h3>
<p>Our app uses FW/1, and I built a role-based security system on top of it. It would require an authenticated session to access controller methods in a cfc with <code>iq:auth=true</code> on the component or on the method in question, and then additionally enforce role restrictions on a per-method basis. Each method could have a metadata attribute named <code>iq:role</code> attached, where a list of roles that would grant you access could be supplied (<code>iq:role="foo,bar,baz"</code>). If you have any one of the roles, you're allowed to proceed.</p>
<p>Alas, Lucee chokes on the colons. I <a href="http://docs.taffy.io/3.2.0#taffydashboardhide">went through this with Taffy</a>, too. I guess I just really like the colons. They remind me of writing ActionScript.</p>
<p>It was easy enough to search/replace to <code>iq_auth</code> and <code>iq_role</code>.</p>
<h3 id="cflocation-script-syntax-difference" tabindex="-1">CFLocation Script syntax difference</h3>
<p>The cfscript version of the <code><cflocation></code> tag is slightly different on Lucee than on ACF. I found the <a href="https://github.com/adamcameron/cfscript/blob/master/cfscript.md">cfscript docs by Adam Cameron</a> helpful in sorting this out.</p>
<h2 id="overall-advice" tabindex="-1">Overall Advice</h2>
<p>In general, be prepared for a lot of confusion and carefully binary-searching your code to find the offending blocks of code:</p>
<p>When you're getting an error for/in one file, comment out the entire file (leaving something otherwise functional, so if it's a CFC comment out everything inside the <code>{}</code> brackets). If the error message <em>changes</em> (or goes away completely) then clearly there's a problem in your commented code. Uncomment about half of it. Did the previous error come back? It's somewhere in that half. If not, then the now-uncommented portion is either fine, or is the source of the 2nd error message. Mark it as "safe" (I used a lot of comments like "everything is fine above this line") and then repeat the process with half of the code that remains commented.</p>
<p>Eventually you should narrow it down to one line or a small chunk of lines. That should hopefully give you some idea of what to look for, and when all else fails ask about it in the CFML Slack. There's a Lucee chat room, and the folks there are always very helpful.</p>
<p>Speaking of which, I need to thank (in the order that they first offered me advice on this journey) Brad Wood, Matthew Clemente, Pete Freitag, and Samuel Knowlton from the #Lucee chat for your help. Thanks, gents!</p>
<p>I imagine I'm not done troubleshooting differences between ACF and Lucee yet, but my app is on its feet.</p>
<h2 id="update%3A-they-keep-helping-me" tabindex="-1">Update: They keep helping me</h2>
<p>I've now also been informed that Lucee supports defining caches in Application.cfc, as seen here:</p>
<p><a href="https://twitter.com/michaelborn_me/status/1171405720987811841">https://twitter.com/michaelborn_me/status/1171405720987811841</a></p>
Two Simple Settings that Make Firefox Usable on OSX2019-06-17T00:00:00Zhttps://adamtuttle.codes/blog/2019/making-firefox-usable-on-osx/<p>Yep, it's kind of a click-bait title. Sorry about that. To make up for it, here's the information you're after, right up front:</p>
<pre><code>gfx.compositor.glcontext.opaque TRUE
browser.tabs.20FpsThrobber TRUE
</code></pre>
<p><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1404042#c167">As suggested here</a>. And don't forget to restart Firefox.</p>
<p>I've been growing more and more uncomfortable with how much I depend on Google on a day to day basis; and <a href="https://www.zdnet.com/article/google-chrome-could-soon-kill-off-most-ad-blocker-extensions/">their announcement of plans to cripple ad-blockers</a> was the last straw. Google appears to be <a href="https://www.zdnet.com/article/google-promises-to-play-nice-with-ad-blockers-again/">trying to walk that back now</a>, but for me at least, the damage is done. I'm working hard to make Firefox work as my go-to browser.</p>
<p>You'd think it would be an easy process... Copy over bookmarks, install a few extensions, and go, right?</p>
<p>Well, gmail works <em>so poorly</em> in Firefox (at least on OSX, at least on my machines -- I tested on two!) that it almost makes me want to believe that it's intentional, anti-competitive behavior. Could it be? Sure. Is it? I have no idea. Would I be surprised? Not at all.</p>
<p>So I did what you do in 2019 when things are crappy: I complained on Twitter about it. And unlike most of my other complaint tweets, I never thought I'd get any help from it. But I did!</p>
<p><a href="https://twitter.com/RandomFFUser/status/1137360604006625280">https://twitter.com/RandomFFUser/status/1137360604006625280</a></p>
<p>Thanks, Random Firefox User.</p>
<p>I'm not actually using a scaled display (as I write this on my hackintosh), though I was originally using a scaled display on my Macbook Pro with its external monitor. It was happening for both. But I figured it was worth a shot and simple enough to revert if it didn't work out.</p>
<p>Since it's mentioned in the post they linked, I should mention that I'm not using a nightly build of Firefox (I'm currently on 67.0.2 in the stable channel). I also didn't read that entire (and very long!) thread. I did have to create the <code>20FpsThrobber</code> setting, but that was easy enough.</p>
<p>If you're wondering how to change these settings, open up a Firefox tab and type <code>about:config</code> in its URL bar. Then search for these settings in the search field at the top of the tab body.</p>
<p>And with that, Firefox is now my go-to browser.</p>
TIL: Running Node.js Apps as a Windows Service2019-05-24T00:00:00Zhttps://adamtuttle.codes/blog/2019/node-apps-as-windows-services/<p>Ah, legacy servers. Gotta love em, right? But just because we have some constraints to work around doesn't necessarily mean we can't have nice things.</p>
<p>As my team transitions to writing more and more of our application as node.js powered microservices, we were starting to feel the pain of not having a good solution for starting things up automatically after an OS reboot. We actually did have a way to make it work, but it involved using the Windows Task Scheduler, and a batch file, and some tedious work any time there was a new node app to add to the mix. Trust me, the world is a better place if I don't share those detials.</p>
<p>Recently a coworker and I went back to the drawing board on this. It's been a few years since we came up with the solution-that-shall-not-be-named, and we figured there's a decent chance that someone figured out something better since then. We're using <a href="https://pm2.io/runtime/">PM2</a> for the process management, and it has built in functionality for daemonizing and restarting your apps after a system reboot, but only for *nix systems. We've known this for a while.</p>
<p>I don't know when, but at some point their official docs added some links to two community projects <strong>pm2-windows-startup</strong> and <a href="https://www.npmjs.com/package/pm2-windows-service">pm2-windows-service</a>. I'm only linking the latter because that's the one we decided to go with.</p>
<p>The former, <strong>pm2-windows-startup</strong> is much simpler in design, with fewer dependencies; but it uses a more blunt approach to get the job done. It installs a registry entry that runs on startup and calls <strong>wscript.exe</strong> (a system app that can run vbscript scripts) and has it run an included vbscript script that then runs the <code>pm2 resurrect</code> command. This doesn't seem super frail to me (certainly a step or three up from our home-rolled solution with task scheduler!) but can we do better?</p>
<p>The latter, <strong>pm2-windows-service</strong> says in its readme that it was inspired by pm2-windows-startup; and it takes a different approach. It installs an actual Windows Service that will start automatically after a reboot, and which you can start/stop/restart from the Services panel. Essentially it accomplishes the same goal: running <code>pm2 resurrect</code> but it also gives you that start/stop/restart control in the same interface as all the rest of your system services. I'm happy to report that this solution is working well for us!</p>
<p>Sadly, it did not work perfectly out of the box, so here's how we were able to get it running:</p>
<p>First, you're going to need pm2, pm2-windows-service, and npm-check-updates (more on that last one in a moment).</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">npm</span> i <span class="token parameter variable">-g</span> pm2 pm2-windows-service npm-check-updates</code></pre>
<p>In theory, your next step would be to run the following:</p>
<pre class="language-bash"><code class="language-bash">$ pm2-service-install</code></pre>
<p>...and answer the questions that it asks. Here's what we did:</p>
<pre class="language-bash"><code class="language-bash">Perform environment setup <span class="token punctuation">(</span>recommended<span class="token punctuation">)</span>? Yes<br /><br />Set PM2_HOME? Yes<br /><br />PM2_HOME value <span class="token punctuation">(</span>this path should be accessible to the <span class="token function">service</span> user and<br />should not contain any <span class="token string">"user-context"</span> variables <span class="token punctuation">[</span>e.g. %APPDATA%<span class="token punctuation">]</span><span class="token punctuation">)</span>: d:<span class="token punctuation">\</span>web<br /><br />Set PM2_SERVICE_SCRIPTS <span class="token punctuation">(</span>the list of start-up scripts <span class="token keyword">for</span> pm2<span class="token punctuation">)</span>? Yes<br /><br />Set the list of start scripts/files <span class="token punctuation">(</span>semi-colon separated json config<br />files or js files<span class="token punctuation">)</span> <span class="token punctuation">(</span>I left this one blank<span class="token punctuation">)</span><br /><br />Set PM2_SERVICE_PM2_DIR <span class="token punctuation">(</span>the location of the global pm2 to use with the <span class="token function">service</span><span class="token punctuation">)</span>? <span class="token punctuation">[</span>recommended<span class="token punctuation">]</span> Yes<br /><br />Specify the directory containing the pm2 version to be used by the<br /><span class="token function">service</span> C:<span class="token punctuation">\</span>USERS<span class="token punctuation">\</span>ADMINISTRATOR<span class="token punctuation">\</span>APPDATA<span class="token punctuation">\</span>ROAMING<span class="token punctuation">\</span>NPM<span class="token punctuation">\</span>node_modules<span class="token punctuation">\</span>pm2<span class="token punctuation">\</span>index.js<br /><br />PM2 <span class="token function">service</span> installed and started.</code></pre>
<p>If that works for you, congratulations, you're basically done! However, it seems that for some more recent versions of node and/or windows, there's a dependency that needs to be updated semi-manually. Here's what to do if you try to run <code>pm2-service-install</code> and it gets hung up after you answer the first question, like this:</p>
<p><img src="https://adamtuttle.codes/img/2019/pm2-service-install-stuck.png" alt="A screen shot of running pm2-service-install and it getting stuck after you answer the first question" /></p>
<p>Fortunately, <a href="https://github.com/jon-hall/pm2-windows-service/issues/51#issuecomment-444160883">other people had already run into this problem and found a solution</a>... Run these steps:</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token builtin class-name">cd</span> %AppData%<span class="token punctuation">\</span>npm<span class="token punctuation">\</span>node_modules<span class="token punctuation">\</span>pm2-windows-service<br />$ ncu inquirer <span class="token parameter variable">-u</span><br />$ <span class="token function">npm</span> i</code></pre>
<p>Here, we're changing into the installation folder for <code>pm2-windows-service</code> and using <code>npm-check-updates</code> (the command for which is <code>ncu</code>) to update package.json with a newer version of the <code>inquirer</code> module. Ours updated from <strong>1.1.2</strong> to <strong>6.3.1</strong>. Then we run npm install to install the updated dependencies. After that, <code>pm2-service-install</code> can run successfully.</p>
<p>Then, how does it work?</p>
<p><code>pm2-service-install</code> creates a windows service that you can control from the services panel. When you start the service, it will use a file located at <code>%PM2_HOME%\dump.pm2</code> to resurrect your running services.</p>
<p>So now all that's left to do is to start each of your apps using pm2 and get everything configured how you'd like, and to run <code>pm2 save</code> to write the state of things into <code>%PM2_HOME%\dump.pm2</code>.</p>
<p>Now, after a system reboot, your node apps will come back online automatically. Just remember to run <code>pm2 save</code> after any change you make to your pm2 state that you want to persist.</p>
The First Hackintosh Milestone: Install Complete2019-05-17T00:00:00Zhttps://adamtuttle.codes/blog/2019/hackintosh-ii-install-complete/<blockquote>
<p>This is part two in an ongoing series where I document my experiences building my first Hackintosh.</p>
<ul>
<li><a href="https://adamtuttle.codes/blog/2019/building-a-hackintosh-2019/">Part one: My motivations, and hardware I purchased</a></li>
<li>Part two: you are here</li>
<li><a href="https://adamtuttle.codes/blog/2020/hackintosh-iii-daily-driver/">Part three: Almost a year later</a></li>
</ul>
</blockquote>
<p>Were you a follower of Joel Spolsky in 2005? Did you follow the <a href="https://www.projectaardvark.com/index.html">Project Aardvark blog</a>, where Fog Creek Software hired a couple of interns to build a product over the summer and document the process? That product went on to become <a href="https://www.copilot.com/">Copilot</a>, a service for providing remote computer-support.</p>
<p>The reason I bring it up is because one of the first few technical posts that went up on their blog -- just 16 days after the blog launched -- was titled, "Reflector: Feature Complete" and I think of it often. The point of the post was that the "feature complete" milestone is just the first of many in the life of a big project.</p>
<p>I've reached that first milestone in my Hackintosh!</p>
<p><img src="https://adamtuttle.codes/img/2019/hackintosh-feature-complete.png" alt="A screen shot of the "system information" dialog immediately after installing OSX on my new hackintosh" /></p>
<p>Of course, there is much yet to do. To give just one example, networking doesn't work yet. There's a whole section of the primary guide that I'm following that is dedicated to required post-install actions, and it might actually be longer than the bios-config and installation part.</p>
<p>So what roadblocks did I have to overcome to get this far? There were a few!</p>
<p>Firstly, the expectation is that you'll use a USB2 (not 3!) thumb drive as your boot/install media, and I didn't have one large enough (16gb) on hand. The only thumb drive I had available >=16gb is USB3. No worries, right? If you plug it into a USB2 port, it should act like a USB2 drive. Maybe? 🤷🏻♂️</p>
<p>It's also recommended that you plug that thumb drive into a USB2 port, if you have any more modern options to choose from. Unfortunately, for whatever reason, none of the USB2 ports on my motherboard would work to boot from the thumb drive... which left me with only USB3 and 3.1 ports to choose from. Fortunately, it seemed like it was working fine (booting the installer) despite using a USB3 thumb drive in a USB3.1 port. 🤷🏻♂️</p>
<p>I have a feeling this entire series of posts is going to make heavy use of the shrug emoji.</p>
<p>It's been more than a few days at this point, so I hope I get this right: What happened next was that the thumb drive would boot initially, but the installer wouldn't start because it couldn't "see" the thumb drive (that is my understanding based on many searches and much reading of the Tonymacx86 forums). In lieu of a helpful error message, I got <a href="https://fontawesome.com/icons/ban?style=solid">a large icon of a circle with a line through it</a>. As it turns out, this was because it couldn't locate the thumb drive, probably because of the USB3 stuff.</p>
<p>I confirmed this by adding a couple of options to the installer from the bootloader screen. When you get to the clover bootloader screen, use the arrow keys to select your thumb drive to boot, and then hit the space bar to open a menu where you can set some boot flags. If you turn on <strong>verbose mode</strong> and <strong>don't reboot on kernel panic</strong> and you should see an error about not being able to find the root device when the "prohibited" icon is displayed, that indicates inability to find the USB thumb drive to continue booting the installer.</p>
<p>Somewhere on the Tonymacx86 forums I saw some advice to try using a USB2 hub to downgrade the thumb drive in this situation. I didn't have a USB2 hub handy, but I did have a USB 1.1 extension cable, so I gave that a shot and it worked. 🎉</p>
<p><img src="https://adamtuttle.codes/img/2019/hackintosh-installer-loading.jpg" alt="The OSX install GUI loading for the first time" /></p>
<p>With the GUI installer now loading, my next task was to prepare (partition, format) the hard drive for the OS to be installed onto. The directions don't explicitly say that <a href="https://www.tonymacx86.com/attachments/screen-shot-2016-09-12-at-9-37-44-am-png.210182/">you should choose the Scheme "GUID Partition Map"</a> (they only mention the Format selection), but that is what's expected and it should be the default. I believe I had found that detail in someone's re-telling of their build on the forum. I ended up switching the Scheme to APFS (more on this later), but I'm not certain it ended up making a difference.</p>
<p>Then things got weird/discouraging.</p>
<p>The installer ran, which is great. Somewhere around "about 2 minutes remaining" the machine rebooted, and I didn't think much of it at the time -- OS installers reboot themselves all the time, right? After the reboot, I saw that the bootloader now showed an option to boot from the SSD and install OSX. That seemed like the obvious next step, so that's what I did. This time it booted into a GUI install and immediately jumped into doing work, with a label of "about 15 minutes remaining." After only a minute or so, it would reboot again. That didn't seem right! (If only I had known...)</p>
<p>Obviously (or so I thought), something was wrong. So I tried again with the verbose/no-reboot-on-panic options selected for the SSD installer boot. Kernel panic! Ugh.</p>
<p>Thinking maybe my USB3 config was still to blame, I decided it was time to bite the bullet and drive to the store (ugh!) and spend a few bucks on a 16gb USB2 thumb drive. Sadly, no improvement.</p>
<p>In desperation, I started searching the forums, and was only able to find one suggestion that seemed worth trying: Switching from GUID Partition Map to APFS. This didn't help anything either. 🤷🏻♂️</p>
<p>It was at this point that I realized I have a very limited and (relatively) infrequent opportunity to troubleshoot this further. I have two kids between 8-10 and it's the time of year where they have lots of school activities and extracurricular things going on. And we weren't going to skip an episode of Game of Thrones... so I probably had 2-3 nights per week, after 9pm, that I could work on it. I figured making a post on the forums and only checking on it a couple of times per week would be bad form, so I decided to put it off until I had more time to focus on it. And there it sat, for about a week.</p>
<p><em><strong>If only I had known!</strong></em></p>
<p>Even though I had to be away from my home office from Friday morning through <em>late</em> Saturday night, I decided I was getting desperate enough to post anyway. On Thursday evening <a href="https://www.tonymacx86.com/threads/need-help-with-10-14-macpro-ish-configuration-total-noob.277365/">I posted on the forum</a> explaining what I had been through so far and asking for help. A few minutes later I went to bed. I checked my email on my phone one last time before turning in for the night, and sure enough someone had replied and explained that the kernel panics were expected and what to do next. Well, there was no going to sleep then!</p>
<p>I returned to my office and got back to work. As it turns out, a couple of (apparent) kernel panics are expected in recent versions of OSX during this process. The advice is to let it restart, continue to choose to boot from the SSD, and let the installer run. I was told it should happen "at least" 3 times. 🤷🏻♂️</p>
<p>Lo and behold, the installer completed and I can now boot into OSX!</p>
<p>I've now reached the section of <a href="https://www.tonymacx86.com/threads/unibeast-install-macos-mojave-on-any-supported-intel-based-pc.259381/">The Guide</a> with the heading "STEP 5: Post Installation with MultiBeast" and that's where I'll pick up on Sunday morning!</p>
That Time I Was Voted Philly's Best Street Performer2019-05-11T00:00:00Zhttps://adamtuttle.codes/blog/2019/phillys-best-street-performer/<p>I live near Philadelphia, and I worked in the city for a previous job. When you go places around the city, especially restaurants, you'll often notice small signs proclaiming the establishment was voted "Philly's Best Pork Roll" or "Philly's Best Coffee" or "Philly's Best Halal Food Truck." I guess these are awarded based on community polling done by one of the local magazines, <a href="https://www.phillymag.com/">PhillyMag</a>.</p>
<p>Imagine my surprise when I got an email one day letting me know that I had won an award from them, and that they wanted my address to send it to me. I had no idea what it was for (the email didn't say), and I figured it was probably a mistake, but</p>
<ol>
<li>I was in a small amount of shock, and</li>
<li>who am I to look a major award in the mouth?!</li>
</ol>
<iframe src="https://giphy.com/embed/3ohs7Mx78Zt2x2xgbu" width="480" height="270" frameBorder="0" class="giphy-embed" allowFullScreen=""></iframe>
<p>So I replied to their email with my address and a note that said I thought they might have found the email address of the wrong Adam Tuttle in Philadelphia. I explained that if they would only let me know what this was all about, I would be happy to set the record straight. I never heard back.</p>
<p>Then one day, this showed up in the mail.</p>
<p><img src="https://adamtuttle.codes/img/2019/phillys-best-street-performer.jpg" alt="Philly's Best Street Performer: Adam Tuttle" /></p>
<p>I don't know who this street performing namesake is, but I am at once grateful for <a href="https://www.phillymag.com/best-of-philly/adam-tuttle/">the good fortune he brings to our shared name</a>, and sad that he might not have ever received his award. I like to imagine that he's still out there drumming on buckets, dreaming of the day he'll be voted Philly's Best. In the meantime, I display this sign proudly in my office.</p>
<p><img src="https://adamtuttle.codes/img/2019/camping-boulder-field.jpg" alt="Our campsite for this weekend, right up against a boulder field!" /></p>
I'm Building a Hackintosh, and I Hope I Don't Regret It!2019-04-30T00:00:00Zhttps://adamtuttle.codes/blog/2019/building-a-hackintosh-2019/<blockquote>
<p>This is part one in an ongoing series where I document my experiences building my first Hackintosh.</p>
<ul>
<li>Part one: You are here</li>
<li><a href="https://adamtuttle.codes/blog/2019/hackintosh-ii-install-complete/">Part two: Hardware build complete, OS Install "feature complete"</a></li>
<li><a href="https://adamtuttle.codes/blog/2020/hackintosh-iii-daily-driver/">Part three: Almost a year later</a></li>
</ul>
</blockquote>
<p>Well... I just spent a bunch of money, so... <em>endorphins!</em></p>
<p><img src="https://adamtuttle.codes/img/2019/hackintosh-parts-order.png" alt="A list of my recent Amazon purchase of computer parts" /></p>
<p>My current work computer is a <strong>Mid 2014 Macbook Pro with a 2.2GHz i7 processor and 16gb of RAM</strong>, and it's been in a long, slow, painful tailspin toward death for months now. I'm actually kind of surprised that it's still alive. I get tons of video artifacts at random intervals. At times certain apps or the whole machine will lock up and I have no choice but to sit and wait for it to catch up. It regularly gets so hot that the glue holding the rubber feet has basically disintegrated, and two of the feet have come off. I keep them on my desk... you know, in case I ever decide to try to reattach them? I know that I never will, but those things are part of <em>my precious</em> so what do you want me to do? Throw them away?!</p>
<p>It's high time that I stop waiting for this thing to die and start getting proactive. But here's the thing... I'm comfortable, but I'm not made of money.</p>
<ul>
<li>It costs about $3,000 for a base-spec Mac Pro: <strong>6 core 3.5GHz, dual gpu, 16gb memory</strong> -- So not that much of an upgrade over my current Macbook Pro!</li>
<li>Or $1,100 for a base-spec iMac: <strong>dual core 2.3GHz, 8gb memory</strong> -- So a literal downgrade from my dying Macbook Pro!</li>
<li>So spending about $1,300 on parts to build a machine capable of eating either of those two options for breakfast is kind of a no-brainer.</li>
</ul>
<p>But I just can't give up OSX. I did an experiment about a month ago where I tried running Ubuntu as my desktop OS on a spare computer and it did not go well. The development experience was acceptable (more or less the same as *nix tools on OSX). But a Chrome update broke Chrome one day, and I don't recall if it was that problem or another one that ended with me re-installing the OS; but the first half of this sentence should give you a pretty clear picture of my experience running Linux as my desktop OS. It was far too fragile for me.</p>
<p>You will have a really hard time as a web developer if you can't use your web browsers.</p>
<p><strong>I need stability.</strong> And I have something like 10+ years experience using OSX as my daily-driver. It's mostly like Windows, with a lot more polish, and dev tools that work well together and don't feel like they were an afterthought (see: PowerShell) or bolted on (see: Windows Subsystem Linux). The core experience of being a developer on OSX has spoiled me.</p>
<p>But Apple hardware? H*ck. I can't afford that stuff!</p>
<p>So Hackintosh here I come. Here's a rundown of what I bought:</p>
<ul>
<li><a href="https://amzn.to/2GTr3JX">Gigabyte Z390 AORUS PRO WiFi</a> motherboard</li>
<li><a href="https://amzn.to/2PG0jiO">Intel Core i7 8700k</a> (6 cores, 4.7GHz)</li>
<li><a href="https://amzn.to/2LelnOS">64gb (4x16gb) memory</a> DDR4 2400 MT/s (PC4-19200)</li>
<li><a href="https://amzn.to/2LbLbLt">Radeon RX580</a> graphics card</li>
<li><a href="https://amzn.to/2LbKEZZ">Cooler Master Hyper 212 Evo</a> CPU Cooler</li>
<li>A pair of <a href="https://amzn.to/2Wg5xEH">Samsung 860 EVO 500GB SSDs</a> to act as boot drives (more on this in a future post)</li>
</ul>
<p>... And I've barely spent more than a base model iMac. I could throw in a <a href="https://amzn.to/2US6VvE">Dell curved 34" 21:9 monitor</a> and <em>still</em> spend less than I would need to configure an iMac for my purposes.</p>
<p>For what it's worth, I'll also be using an ATX case, 550 watt power supply, and some 2tb hard drives, all of which I already had on hand.</p>
<p>To be honest, I'm a little terrified that I'm dropping a huge pile of cash on a gamble that I can pull off a functioning Hackintosh. I've wanted to do this for years and years but never done it before, and the idea of failing with such a high price tag is scary. So yeah, I'm a lotta terrified. But lots of people have been doing this for a long time, and it's getting easier all the time, right?</p>
<p>So let's do this... I'll do my best to chronicle my experience here, so stay tuned!</p>
<p>And who knows... If this works out well, maybe I'll treat myself to a victory monitor...</p>
<p><a href="https://amzn.to/2US6VvE"><img src="https://adamtuttle.codes/img/2019/Dell_U3415W.jpg" alt="A picture of the Dell 34 inch curved 21:9 monitor that I'm drooling over" /></a></p>
Friction Stops Things2019-04-16T00:00:00Zhttps://adamtuttle.codes/blog/2019/friction-stops-things/<p>Oh my. I haven't written a blog post since November 2017. I wonder why that is. 🤔</p>
<p><a href="https://twitter.com/mipsytipsy/status/1117858830136664067">https://twitter.com/mipsytipsy/status/1117858830136664067</a></p>
<p>I guess it's not a direct parallel, but this tweet<sup class="footnote-ref"><a href="https://adamtuttle.codes/blog/2019/friction-stops-things/#fn1" id="fnref1">[1]</a></sup> really struck a chord with me, in more areas than just the Friday-deployments problem space mentioned. Friction stops things.</p>
<p>I converted my old dynamically rendered blog from a CFML platform to a static site generator because because I wanted a place where I could experiment with something new and interesting, and because static site generators were all the rage. I figured if it was as simple as compiling and pushing to GitHub, that would be pretty low-friction, and I would be able to write more often.</p>
<p>I think we can safely conclude that the experiment was a failure.</p>
<p>Normally I wouldn't think that saying, "Friction Stops Things" would even be worth saying, but hey... If the President of the United States of America (a Stable Genius, I'll remind you) is throwing out <a href="https://twitter.com/realDonaldTrump/status/1117844987293487104">"Maybe they should put some water on that fire"</a> as advice, maybe this level of insight could actually prove useful to someone.</p>
<p>Obviously the tooling I was using for the previous iteration of my static-site blog was still too high-friction. Truth be told, running it on GitHub Pages meant that I had to run the compile step, change into the generated content folder, which was a clone of the <code>gh-pages</code> branch of the same repo, and then commit and push those changes. Huge pain? No, of course not. But I guess my laziness knows no bounds.</p>
<p>Which leads me to the new tooling: Gatsby and Netlify. I've been hearing nothing but heaps of praise for these things and if I understand correctly, all that I will need to do to post a new entry is to create and commit/push a new markdown file.</p>
<p>Easier. Easy enough? I guess we'll see.</p>
<hr class="footnotes-sep" />
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>It's more than just one good tweet, actually. It's a fantastic Twitter thread; and in some ways it has parallels to what I'm getting into here, but I thought it would be too distracting to pull them all in. But seriously, go read the whole thread! <a href="https://adamtuttle.codes/blog/2019/friction-stops-things/#fnref1" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
GraphQL After Tutorials: The Blog Post I Wish I Had when I Started2017-11-06T00:00:00Zhttps://adamtuttle.codes/blog/2017/graphql-after-tutorials/<p>There is a ton of great getting started content out there for you to, well, get started with GraphQL. For starters, I recommend <a href="https://www.howtographql.com/">How To GraphQL</a>. And if your server platform of choice is node.js, I also recommend <a href="https://github.com/apollographql/apollo-server">Apollo Server</a> because it does a tremendous job of getting rid of boilerplate and letting you focus on your schema and resolvers. Of course, if you're using React for your front-end, they also make a pretty great GraphQL client: <a href="https://github.com/apollographql/apollo-client">Apollo Client</a>.</p>
<p>But what I wanted to write about here is all of the things that were going on in my head after I finished the getting started tutorials. So I'm going to assume you've got a little bit of your schema and resolvers written and they're working well.</p>
<p>But what about related data types from my schema? What about authentication? Action-authorization? Error logging? How does my client get the inserted id back from a create mutation?</p>
<h2 id="related-data" tabindex="-1">Related Data</h2>
<p>This one is specific to Apollo Server. You have a <code>User</code> type and a <code>Role</code> type, and users can have Roles:</p>
<pre class="language-js"><code class="language-js">type Role <span class="token punctuation">{</span><br /> <span class="token literal-property property">roleId</span><span class="token operator">:</span> Int<span class="token operator">!</span><br /> <span class="token literal-property property">name</span><span class="token operator">:</span> String<br /><span class="token punctuation">}</span><br /><br />type User <span class="token punctuation">{</span><br /> <span class="token literal-property property">userId</span><span class="token operator">:</span> <span class="token constant">ID</span><span class="token operator">!</span><br /> <span class="token literal-property property">name</span><span class="token operator">:</span> String<br /> <span class="token literal-property property">roles</span><span class="token operator">:</span> <span class="token punctuation">[</span>Role<span class="token punctuation">]</span><br /><span class="token punctuation">}</span></code></pre>
<p>If you're using a traditional SQL database then you likely have a table that defines that many-to-many relationship. How do you query that?</p>
<p>There's a special place to put that resolver:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> resolvers <span class="token operator">=</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">Query</span><span class="token operator">:</span> <span class="token punctuation">{</span><span class="token operator">...</span><span class="token punctuation">}</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">User</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token function">roles</span><span class="token punctuation">(</span><span class="token parameter">obj</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">return</span> roleService<span class="token punctuation">.</span><span class="token function">getUserRoles</span><span class="token punctuation">(</span>obj<span class="token punctuation">.</span>userId<span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">}</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">}</span><span class="token punctuation">,</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token literal-property property">Mutation</span><span class="token operator">:</span> <span class="token punctuation">{</span><span class="token operator">...</span><span class="token punctuation">}</span></mark><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>The <code>obj</code> arg passed to this <code>User.roles</code> resolver is the User record after it gets resolved.</p>
<blockquote>
<p><strong>Note:</strong> This is not the most efficient approach, using two separate queries. I've heard rumors that it's possible to "look ahead" at the incoming query and detect certain situations to fork off to a more efficient single-query approach. I haven't figured that out yet.</p>
</blockquote>
<h2 id="authentication" tabindex="-1">Authentication</h2>
<p>The general line here is, "don't reinvent the wheel." Whatever you're using for the rest of your application should be fine for the API too. This took me some time to wrap my head around since I've spent years adding token-based auth to REST APIs.</p>
<p>In my case, I've got an Express-powered web app using <code>express-session</code> and <code>connect-redis</code> to load people's sessions. There are a couple of different ways to use Apollo Server but to hook into your Express Session plugin setup, you need to use <code>graphqlExpress</code> (provided in <code>apollo-server</code>) to create an Express instance and then hook it up to Apollo Server. In the options for graphqlExpress you can pass a <code>context</code> object, to be passed to all resolvers.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">server<span class="token punctuation">.</span><span class="token function">use</span><span class="token punctuation">(</span></span><br /><span class="highlight-line"> <span class="token string">'/graphql'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> bodyParser<span class="token punctuation">.</span><span class="token function">json</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token function">graphqlExpress</span><span class="token punctuation">(</span><span class="token parameter">request</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">return</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> schema<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token comment">//make req user available to all resolvers</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token literal-property property">context</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token literal-property property">user</span><span class="token operator">:</span> request<span class="token punctuation">.</span>session<span class="token punctuation">.</span>user <span class="token punctuation">}</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">}</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Of course this assumes that you've already created <code>session.user</code> -- the same way that you would for any other web app.</p>
<p>This <em>does not</em> reject requests that aren't logged in. Just like a regular express app, that would be up to you too. The usual approach for your web app is to redirect to the login screen. For REST APIs the usual approach is to return a 401, which the client understands to mean that the user isn't authenticated. I don't know if that's the official solution for GraphQL APIs too, but it's what I'm doing:</p>
<pre class="language-js"><code class="language-js">server<span class="token punctuation">.</span><span class="token function">use</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span> res<span class="token punctuation">,</span> next</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>req<span class="token punctuation">.</span>session<span class="token punctuation">.</span><span class="token function">hasOwnProperty</span><span class="token punctuation">(</span><span class="token string">'user'</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> res<span class="token punctuation">.</span><span class="token function">status</span><span class="token punctuation">(</span><span class="token number">401</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">end</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<h3 id="action-authorization" tabindex="-1">Action-Authorization</h3>
<p>Now that I know the person making the request is logged in, what if I want to make sure they don't do things they shouldn't have access to?</p>
<p>Let's assume you have a service method that is used to save updates to user accounts, but in order to use it you want to make sure the user has a specific role, <code>save-user</code>.</p>
<p>The first thing you need to do is get that session user from the resolver's context argument and pass it to the service method:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> resolvers <span class="token operator">=</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">Query</span><span class="token operator">:</span> <span class="token punctuation">{</span><span class="token operator">...</span><span class="token punctuation">}</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">Mutation</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token function-variable function">saveUser</span><span class="token operator">:</span> <span class="token punctuation">(</span><span class="token parameter">root<span class="token punctuation">,</span> args<span class="token punctuation">,</span> context</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">return</span> userService<span class="token punctuation">.</span><span class="token function">saveUser</span><span class="token punctuation">(</span>context<span class="token punctuation">.</span>user<span class="token punctuation">,</span> args<span class="token punctuation">.</span>user<span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">}</span></mark><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>And then you need to enforce the role requirement:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> <span class="token function-variable function">saveUser</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">reqUser<span class="token punctuation">,</span> userToSave</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>reqUser<span class="token punctuation">.</span>roles<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">'save-user'</span><span class="token punctuation">)</span> <span class="token operator">===</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">throw</span> <span class="token keyword">new</span> <span class="token class-name">Error</span><span class="token punctuation">(</span><span class="token string">'Access Error'</span><span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">}</span></mark><br /><span class="highlight-line"> <span class="token comment">//save userToSave...</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<h2 id="error-logging" tabindex="-1">Error Logging</h2>
<p>This one turns out to be pretty easy. What you do with the errors is up to you (write to a log file, push them out to a service like Raygun, etc). We just need to give Apollo Server a function that does what we want with the exception:</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line">server<span class="token punctuation">.</span><span class="token function">use</span><span class="token punctuation">(</span></span><br /><span class="highlight-line"> <span class="token string">'/graphql'</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> bodyParser<span class="token punctuation">.</span><span class="token function">json</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token function">graphqlExpress</span><span class="token punctuation">(</span><span class="token parameter">request</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token keyword">return</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> schema<span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token function-variable function">formatError</span><span class="token operator">:</span> <span class="token parameter">err</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> console<span class="token punctuation">.</span><span class="token function">error</span><span class="token punctuation">(</span>err<span class="token punctuation">.</span>stack<span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token literal-property property">context</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token literal-property property">user</span><span class="token operator">:</span> request<span class="token punctuation">.</span>session<span class="token punctuation">.</span>user <span class="token punctuation">}</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<h2 id="getting-inserted-id-from-mutation" tabindex="-1">Getting inserted id from mutation</h2>
<p>Apollo client gives us a really powerful <code>mutate</code> method (<a href="https://www.apollographql.com/docs/react/basics/mutations.html#calling-mutations">docs</a>), but of course with power comes a bit of complexity. One common necessity that I ran into and had some trouble with, even after following a tutorial that showed me exactly this, was getting the inserted id value back from a mutation that creates a database row.</p>
<p>Let's say we're creating a new user and after it's saved we want to redirect to the view/edit screen for the new record. The mutate options object can contain an <code>update</code> method that gets called with the response from the mutation. So if your createUser mutation returns the created user, you might do the following:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token function">mutate</span><span class="token punctuation">(</span><span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">variables</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token literal-property property">user</span><span class="token operator">:</span> userToSave <span class="token punctuation">}</span><span class="token punctuation">,</span></span><br /><span class="highlight-line"> <span class="token function-variable function">update</span><span class="token operator">:</span> <span class="token punctuation">(</span><span class="token parameter">store<span class="token punctuation">,</span> <span class="token punctuation">{</span> <span class="token literal-property property">data</span><span class="token operator">:</span> user <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token function">redirectToEditUser</span><span class="token punctuation">(</span>user<span class="token punctuation">.</span>userId<span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">}</span></mark><br /><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<h2 id="what's-next%3F" tabindex="-1">What's next?</h2>
<p>This post represents most of what I've learned in the last week or two as I've gotten the first few screens of my app ported over to GraphQL.</p>
<p>What's next? Honestly, I don't really know! <em>Unknown unknowns.</em> Eventually I want to figure out that more efficient lookup approach I mentioned in the <strong>Related Data</strong> section. But I'm sure I still have much to learn about GraphQL, so expect more tips in the future...</p>
<p>Did this post help you out? <a href="https://twitter.com/AdamTuttle">Send me a shout-out on Twitter</a>. I'd love to hear from you.</p>
Stupid CFML Tricks: Run After Return2017-05-12T15:30:00Zhttps://adamtuttle.codes/blog/2017/stupid-cfml-tricks-run-after-return/<blockquote>
<p>This article was originally focused on CFML and erroneously claimed that you could execute code after a <code>return</code> statement by putting it in the <code>finally</code> block of a <code>try/finally</code> statement. What follows is what I've since learned about how try/catch/finally works. Thanks to <a href="https://corfield.org/">Sean Corfield</a> for educating me on how I originally understood this incorrectly.</p>
</blockquote>
<p>When I first learned about try/catch/finally, everything made perfect sense to me. The <code>try</code> block executes; if there's an error it's handled by the <code>catch</code> block, and then if there's a <code>finally</code> block then it is executed after the <code>try</code> block (if successful) or after the <code>catch</code> block (if an error was thrown in the <code>try</code> block).</p>
<p>Now let's make things interesting! What happens when your try block contains a <code>return</code> statement?</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">const</span> <span class="token function-variable function">example</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><br /> <span class="token keyword">let</span> x <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">;</span><br /> <span class="token keyword">try</span> <span class="token punctuation">{</span><br /> <span class="token keyword">return</span> x<span class="token punctuation">;</span><br /> <span class="token punctuation">}</span> <span class="token keyword">finally</span> <span class="token punctuation">{</span><br /> x <span class="token operator">=</span> <span class="token number">2</span><span class="token punctuation">;</span><br /> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">'inside finally! x='</span><span class="token punctuation">,</span> x<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><span class="token punctuation">;</span><br /><br />console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token function">example</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>Without testing it, what do you think happens? Does the function return 1 or 2?</p>
<p>As it turns out, the function returns 1; but first it logs that x=2.</p>
<p>This makes sense because the sequence of events is:</p>
<ul>
<li><code>x = 1</code></li>
<li>the line <code>return x</code> is evaluated; but...</li>
<li>since there's a <code>finally</code> block, that's executed, so...</li>
<li>x is then updated to the value 2,</li>
<li>it prints <code>inside finally! x=2</code></li>
<li>and now that the finally block has executed, the <code>return x</code> (which was evaluated to <code>return 1</code>) executes.</li>
</ul>
<p>Using <code>finally</code> is shorthand for copy+pasting the contents of the finally block to precede all possible exit statements (return, throw) inside the <code>try</code> block.</p>
Adding SSH Passphrase to Your Keychain on macOS Sierra2016-12-19T08:00:00Zhttps://adamtuttle.codes/blog/2016/Adding-SSH-passphrase-to-keychain-on-macOS-Sierra/<p class="photo-byline"><img src="https://adamtuttle.codes/img/2016/scott-webb-lnRPKo7Lo5Q-unsplash.jpg" alt="Keys hanging from a lego brick used as a keychain attached to some lego bricks on the wall" />
Photo by <a href="https://unsplash.com/@scottwebb?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Scott Webb</a> on <a href="https://unsplash.com/s/photos/keychain?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>The truth is that I just don't have anywhere better to jot this down for my own future reference, so I'm inflicting it on the world as another blog post. I guess it could prove helpful to some people too, so there's that.</p>
<p>Due to <a href="https://motherboard.vice.com/read/this-300-device-lets-you-steal-a-mac-encryption-password-in-30-seconds">a recent security announcement</a>, I decided it was time to stop putting off my upgrade to Sierra.</p>
<p><a href="https://twitter.com/AdamTuttle/status/809832821045006336">https://twitter.com/AdamTuttle/status/809832821045006336</a></p>
<p>After dinner that night I forgot to make sure my latest Apache config changes were checked into git and took the plunge. A couple of hours later and I was back on my feet. For what it's worth, not much has changed in the day to day operation of my computer, so that's good.</p>
<p>However, one thing I noticed quickly –aside from the fact that Apple brazenly threw out anything they deemed unnecessary in my Apache config, as they always do– was that I was now required to enter my SSH passphrase every time I use ssh.</p>
<p><img src="https://adamtuttle.codes/img/2016/ssh-passphrase.jpg" alt="SSH passphrase required" /></p>
<p>In general I would say this is a good thing. I am pro-security. But I use ssh for my git repos and I'm pushing and pulling commits all day long. Entering my password 50+ times in a day doesn't sound fun. I use whole-disk encryption, a strong system password, require my password immediately after the screen saver kicks on, and have developed a healthy habit of throwing my mouse cursor into the hot-corner that activates the screen saver if I have to leave my laptop unattended.</p>
<p>Also, I work from home so I more frequently go outside in my slippers than leave my laptop on a desk in an office while I use the restroom. (I estimate this at about a 200:1 ratio. I really like my slippers.)</p>
<p>While conceding that it is <em>slightly less secure</em> now, I also feel that requiring it so often before was overkill. I asked on Twitter if anyone had advice, and as it so often does, <a href="https://twitter.com/AdamTuttle/status/810117274355040256">the internet delivered</a>.</p>
<p>First, <a href="http://superuser.com/questions/1127067/macos-keeps-asking-my-ssh-passphrase-since-i-updated-to-sierra">add the key to the keychain</a>:</p>
<pre class="language-bash"><code class="language-bash">$ ssh-add <span class="token parameter variable">-K</span> /Users/atcodes/.ssh/id_rsa</code></pre>
<p>Note that the absolute path to the key file is used, not <code>~/.ssh/id_rsa</code>.</p>
<p>Then, <a href="https://github.com/jirsbek/SSH-keys-in-macOS-Sierra-keychain">add this file</a> as <code>~/Library/LaunchAgents/ssh.add.a.plist</code>:</p>
<pre class="language-xml"><code class="language-xml"><span class="token prolog"><?xml version="1.0" encoding="UTF-8"?></span><br /><span class="token doctype"><span class="token punctuation"><!</span><span class="token doctype-tag">DOCTYPE</span> <span class="token name">plist</span> <span class="token name">PUBLIC</span> <span class="token string">"-//Apple//DTD PLIST 1.0//EN"</span> <span class="token string">"http://www.apple.com/DTDs/PropertyList-1.0.dtd"</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>plist</span> <span class="token attr-name">version</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>1.0<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>dict</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>key</span><span class="token punctuation">></span></span>Label<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>key</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>string</span><span class="token punctuation">></span></span>ssh-add-a<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>string</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>key</span><span class="token punctuation">></span></span>ProgramArguments<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>key</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>array</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>string</span><span class="token punctuation">></span></span>ssh-add<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>string</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>string</span><span class="token punctuation">></span></span>-A<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>string</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>array</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>key</span><span class="token punctuation">></span></span>RunAtLoad<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>key</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>true</span><span class="token punctuation">/></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>dict</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>plist</span><span class="token punctuation">></span></span></code></pre>
<p>You can create it from your terminal with this one-liner: <code>curl -o ~/Library/LaunchAgents/ssh.add.a.plist https://raw.githubusercontent.com/jirsbek/SSH-keys-in-macOS-Sierra-keychain/master/ssh.add.a.plist</code></p>
<p>This adds a startup task that will run <code>ssh-add -A</code> every time you restart your computer.</p>
<p>Now your SSH passphrase isn't required quite so often.</p>
Extra-Life 20162016-11-15T10:00:00Zhttps://adamtuttle.codes/blog/2016/extra-life-2016/<p>It is that time of year again! Time for me to beg you for donations to, in my opinion, one of the most honorable causes there is: The wellbeing of children.</p>
<p>For the uninitiated, Extra-Life is a charity that raises money for <a href="https://chop.childrensmiraclenetworkhospitals.org/">Children's Miracle Network Hospitals</a>, who support children in the hospital, often for scary stuff like a heart condition or cancer, when their families can't. All of the money that I raise gets earmarked for the <a href="http://www.chop.edu/">Children's Hospital of Pennsylvania</a>, headquartered here in Philadelphia and serving the entire state.</p>
<p>Donations are tax-deductible and secure, and Children's Miracle Network is so transparent and responsible with how they spend collected donations that they are <a href="http://www.charitynavigator.org/index.cfm?bay=search.summary&orgid=5756">rated 4/4 stars on Charity Navigator</a>.</p>
<p>I believe it was last year around this time that my niece had open heart surgery. She just had another one last week. She has an artificial heart valve and a pacemaker. She's still at CHOP recovering. <em>She's 4 years old.</em> She is fortunate enough to have been adopted by a family with good health insurance; but not everyone is. And that is why I do this every year. For kids who have done nothing wrong but who are born with a heart defect, or get cancer, or get hurt in a car accident. They deserve our love and our support, and the healthcare system in this country will sideline them if we don't step up to the plate.</p>
<p>So I play games on their behalf in the hope that you will skip a cup of coffee and donate a few dollars instead. Sure it's fun, but it's also a constant reminder of how fragile life is and how lucky we are to be alive right now.</p>
<h2 id="get-something-in-return" tabindex="-1">Get something in return</h2>
<p>This year I thought I would sweeten the pot, too:</p>
<p><a href="http://restassuredbook.com/"><img src="https://adamtuttle.codes/img/2016/rest-assured-extra-life.png" alt="Flyer for my book deal" /></a></p>
<p>From now until we start playing (that's 8:00 am US-Eastern on 11/19), 100% of profits earned through the <a href="http://restassuredbook.com/">sales of my book</a> will be donated to my campaign. The book costs $19 and my cut –$17.80– will go to charity.</p>
<h2 id="streaming-live%3F" tabindex="-1">Streaming Live?</h2>
<p>In recent years I've spent the majority of the time on a Google Hangout, but I thought I might try streaming it live on Facebook this year.</p>
The Cost of Abstraction2016-10-07T11:00:00Zhttps://adamtuttle.codes/blog/2016/the-cost-of-abstraction/<p>Earlier today a friend and colleague who is almost 40 sent me a link to this article: <a href="http://www.bennorthrop.com/Essays/2016/reflections-of-an-old-programmer.php">Reflections of an "Old" Programmer</a>. I'm 34, so still a few years shy of Ben, but I have been starting to get the feeling that I'm an "old" coder too. So hopefully my thoughts in response aren't too far off-base.</p>
<p>I think the motivation for Ben's post is correct:</p>
<blockquote>
<p>Without ... effort, we could be worse at our jobs in 5 years than we are today. There is no coasting.</p>
</blockquote>
<p>But his metaphors are full of holes and his thesis of knowledge decay is too broad. Allow me to explain.</p>
<h2 id="metaphors-revisited" tabindex="-1">Metaphors revisited</h2>
<p><a href="https://en.wikipedia.org/wiki/Ignaz_Semmelweis">Ignaz Semmelweis</a>, a Hungarian doctor in the mid 1800s, was ridiculed because he believed that microscopic living organisms could reside on your skin and be transmitted from one patient to the next, spreading disease. Of course today we understand about bacteria and viruses and we wash our hands between surgeries.</p>
<p>Ben asserts that doctors (probably) don't find themselves constantly dealing with knowledge decay and the threat of getting worse at their jobs over time:</p>
<blockquote>
<p>The doctor at 40 doesn't seem to be worried about discovering that all his knowledge of the vascular system is about to evaporate in favor of some new organizing theory. The same goes for the lawyer, the plumber, the accountant, or the english teacher.</p>
</blockquote>
<p>While there is a nugget of truth in this quote (more on that in a moment), I would assert in response that Ben probably doesn't know too many doctors, lawyers, plumbers, accountants, or teachers. Or at least he doesn't ask them about their Continuing Learning Efforts (CLE's).</p>
<p>Your family doctor probably attends at least one conference a year, and very likely <em>regularly</em> attends lunch-and-learn sessions paid for by pharmaceutical companies where she gets free lunch in exchange for learning about the company's new drugs: How they work, what they do, and what off-label uses they might be good for, to name just a few things. I have friends that were pharma reps and friends that are doctors, and I'm speaking from first-hand conversations with them.</p>
<p>Anyone remember "New Math"? Ben's parents were probably complaining about it when he was in 6th-10th grade, if my math is correct. Teachers were learning and applying new techniques for helping kids to understand mathematical concepts. The same is probably true for reading, writing, and other concepts too. And we're going through it again now: What people often refer to as "common core" (which I support) –which is actually just a new program put in place to <em>satisfy</em> common core– is an updated approach to teaching based on scientific research on the subject.</p>
<p>Lawyers must be aware of new laws and precedents. Accountants have to keep up with the latest loop-holes to help their billionaire clients avoid paying taxes. Plumbers need to know about new materials (my house is plumbed with PEX pipes) and tools.</p>
<p>In all of these cases, if someone abandons learning new things the day they get their first job they will not have a very long career at all.</p>
<h2 id="that-nugget-of-truth" tabindex="-1">That nugget of truth</h2>
<blockquote>
<p>The doctor at 40 doesn't seem to be worried about discovering that all his knowledge of the vascular system is about to evaporate in favor of some new organizing theory.</p>
</blockquote>
<p>This part is right, though. The vascular system is well understood and documented, and the hardware is unlikely to change drastically over the course of a doctor's career. Except artificial blood vessels and hearts and whatever other medical innovations come along in the next 40 years...</p>
<p>There are certain concepts in any field, which I'll refer to as "core concepts," that have a half life of much longer than the average human lifespan, if not effectively infinite.</p>
<p>Looping, Branching, Recursion: These are all programming core concepts that you will not likely find yourself abandoning for new approaches in our lifetime. Not unless there are major breakthroughs in Quantum Computing, followed by mass market production of quantum computers to make them as common as PCs are today. These are the vascular and nervous systems of the computer: You're using concepts born at a lower level of abstraction, even if you're using a 4th or 5th generation language.</p>
<p>In the beginning there were 1st generation languages: byte code punched into cards. Then there were 2nd generation languages that abstracted a collection of bytecode commands into a single command that could be written with letters and numbers in a text file, i.e. Assembly. 3rd generation languages like C abstract collections of 2GL commands into a single line of code. And repeat for 4th and 5th generation languages.</p>
<p>Each layer of abstraction exists because it "makes hard things easier" than the previous generation. But we're at a point now where we've based our knowledge on something 4 or 5 layers of abstraction away from implementation. And because of that, things change more frequently.</p>
<p>The problems that these JavaScript frameworks du jour are attempting to solve are simply not solved: The scientists (programmers) are still experimenting with different methods and techniques, and no consensus has been reached on the best ways to tackle these problems. Nor is it likely to be in our lifetime. So in this limited scope, I agree with Ben: the half-lives of these things are low when contrasted with core concepts.</p>
<p>It is these higher layers of abstraction that are changing, not the core concepts. When you switch to a new tool or framework, recursion still works the same way it used to.</p>
<p>You can't compare your highly abstracted concepts to the core concepts of another profession and proclaim that our industry is just different. That we churn more knowledge. We don't. Drugs get discontinued. Lead paint and leaded gasoline (and if you're me, you're hoping gasoline altogether!) are on their way out. Every industry evolves.</p>
<p><strong>This is the cost of working at a high level of abstraction.</strong></p>
<h2 id="the-result-is-the-same" tabindex="-1">The result is the same</h2>
<p>Even with all of the above nit-picking, I agree with Ben's conclusion: Focus on learning the core concepts of your field well and they will serve you well for a long time to come. For everything else, don't rush to be an early adopter, and wait for some amount of consensus to grow before you dive in head-first.</p>
<p>Early adoption is for "kids" (pre-30's) that have time to be burned by bad decisions and recover from those burns. Us old-folks need to focus on stability and sustainability.</p>
<p>But I still think React Native looks pretty neat.</p>
Giggle Driven Development2016-09-30T09:00:00Zhttps://adamtuttle.codes/blog/2016/giggle-driven-development/<p>Yesterday morning while spending time with my kids before they left for school, and completely by accident, I found myself on a website making use of some really neat technology that made programming problems and lego-block style "coding" of solutions really novel, simple, and approachable. We had a bunch of fun working on some of the problems together, and I feel compelled to share.</p>
<p><img src="https://adamtuttle.codes/img/2016/giggle-fail-algorithm.png" alt="Sample (wrong) source code" /></p>
<p>The technology is called <a href="https://developers.google.com/blockly/">Blockly</a> (by Google), and <a href="https://blockly-games.appspot.com/">these are the games we played</a>.</p>
<p>The games start out with a simple puzzle mechanic that teaches you how to snap the puzzle pieces (that will later become "code") together, and then moves on to the most basic form of coding challenges. The challenges get successively harder as you go, requiring the understanding of a new concept to complete each level.</p>
<p>This feels like a very natural next step after <a href="http://www.robotturtles.com/">Robot Turtles</a>, a board game that teaches beginning programming concepts. I backed it on kickstarter and my kids and I enjoy breaking it out from time to time. Two things that Robot Turtles lacks and that Blockly introduces pretty early on are branching (if/else) and looping constructs. Those concepts would be really hard to model out with a deck of cards laid out on a table – the mechanic used in Robot Turtles – but the snap-together pieces approach in Blockly actually works pretty well.</p>
<p><img src="https://adamtuttle.codes/img/2016/giggle-fail.jpg" alt="A screen shot of one of our failed attempts, which elicited much laughter" /></p>
<p>But the thing that really sucked my kids in was what happens when you fail. Instead of just making a "you can't do that" noise and stopping the little maze-wanderer dead in its tracks, the astronaut flies off into space. If you're playing with the default skin that resembles Google Maps then it sounds like the character walks into a wall. Hard. And if you're using the panda theme, it makes some sort of "mew" noise and then falls off of the bottom of the screen.</p>
<p>That accidental joy: turning a failure into laughter; that's what drew my kids in and made them want to do more. They sought out more failures, and in doing so, learned more about the concepts being taught. Not because they wanted to learn more about programming, but because they thought it was hilarious when the astronaut floated off into space and they wanted to understand how to make it happen again, intentionally.</p>
<p><img src="https://adamtuttle.codes/img/2016/giggle-scratch.png" alt="A screen shot of MIT Scratch" /></p>
<p>In the past, I've tried introducing them to <a href="https://scratch.mit.edu/">MIT's Scratch</a>, but it never seemed to interest them. From what I've seen so far, Scratch can be quite a bit more complex, though the only thing I've seen done with it is to make (sometimes interactive) animations. Maybe once they get some more time in with Blockly they'll be more interested in Scratch. Or maybe my kids are puzzle gamers, and just not interested in creating animations.</p>
<p>But clearly the hilarious (to a 5 and 7 year old) way that failure happens in these Blockly games is part of what makes them so appealing.</p>
<p>And what can we learn from this as we create interfaces for non-technical users in business applications? We aren't teaching them to write code, but if we can turn their mistakes into something that makes them <em>want</em> to spend more time using the application, I would consider that a win.</p>
<p>How cool would it be to have some negative behavior in your app enable <a href="https://github.com/codeinthedark/awesome-power-mode">Power Mode</a> instead of some boring "You can't do that" validation? I don't know when it would be appropriate to do so (outside of, perhaps, April Fools) but it strikes me as the type of thing that is harmless and would elicit joy from the users, if used sparingly.</p>
<small>
<a href="https://www.flickr.com/photos/philiproeland/12100005003">
Photo credit
</a>
</small>
My Experience With AWS Reserved EC2 Instances and Deploying Docker in Production2016-09-06T09:00:00Zhttps://adamtuttle.codes/blog/2016/AWS-Reserved-and-Docker-in-production/<p class="photo-byline"><img src="https://adamtuttle.codes/img/2016/paul-teysen-bukjsECgmeU-unsplash.jpg" alt="Shipping containers stacked high" />
Photo by <a href="https://unsplash.com/@hooverpaul55?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Paul Teysen</a> on <a href="https://unsplash.com/s/photos/containers?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>This is going to be more field-notes than how-to, but when I tweeted about how I had such an easy experience getting some Docker containers running in production I had several people express interest in reading about my experiences, so here we are.</p>
<p>The backstory is that I had one app in particular that was on a terrible, terrible web host (like really, the worst) –and it was for a paid client, at that– so when the host spent more time offline than online in June, and ignored my numerous support tickets, I decided to take my money elsewhere. The app happened to be a fairly simple website for a non-profit organization, with a members-only area, a way to pay dues and register for events, and a few other odds and ends. On the host-that-shall-remain-nameless this app was running on Adobe ColdFusion 10, and using a local MariaDB. I am not willing to pay the price Adobe charges for their AMIs long term, so I knew I had to convert to Lucee.</p>
<p>Having recently attended the <a href="http://www.devobjective.com/">dev.Objective()</a> conference and subsequently been <a href="http://adamtuttle.codes/TIL-adding-a-jvm-ssl-cert-docker/">playing with Docker</a> I decided I would use it to develop the necessary changes. It really made getting a development environment up and running terribly easy. In fact, I've made it this far in my life without <em>properly</em> learning how to install and configure Tomcat, why start now?</p>
<p>And truly, that's the best part about all of this: <strong>Standing on the shoulders of giants.</strong> My most sincere thanks go to the Lucee team, in particular Geoff Bowers who manages their docker images. Also many thanks to the fine folks in the #lucee and #docker channels on the <a href="http://cfml-slack.herokuapp.com/">CFML Slack</a> who helped me through issues as they would come up.</p>
<p><video src="https://adamtuttle.codes/img/2016/first-water-landing-720p.webm" controls="">Sorry, your browser doesn't support embedded videos, but don't worry, you can <a href="https://adamtuttle.codes/img/2016/first-water-landing-720p.webm">download it</a> and watch it with your favorite video player!</video></p>
<p>Right, so after a month of evenings and weekends toiling away at the code changes, I got my local docker container for my app into good shape. I also managed to get a separate container up with MariaDB and network the two together. It worked, but it wasn't pretty. I had a series of shell scripts to stand it up and hold it together, but nothing I would want to use in production. Most importantly, I had no idea how I would make the DB survive a container restart (it was building from scratch on every start).</p>
<p>For the former problem (my poorly written shell scripts) <a href="https://docs.docker.com/compose/">docker-compose</a> came to the rescue, and for the latter (DB data storage) all I had to do was RTFM. Compose makes it easy to define a set of containers that need to be started and networked together, and manage them. And <a href="https://hub.docker.com/_/mariadb/">the MariaDB docker image docs</a> literally have a heading, "Where to Store Data."</p>
<p>I'm not going to spell out how those things work in detail, but here's my <code>docker-compose.yml</code> file:</p>
<pre class="language-yaml"><code class="language-yaml"><span class="highlight-line"><span class="token key atrule">version</span><span class="token punctuation">:</span> <span class="token string">'2'</span></span><br /><span class="highlight-line"><span class="token key atrule">services</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token key atrule">web</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token key atrule">container_name</span><span class="token punctuation">:</span> myapp_web</span><br /><span class="highlight-line"> <span class="token key atrule">restart</span><span class="token punctuation">:</span> always</span><br /><span class="highlight-line"> <span class="token key atrule">build</span><span class="token punctuation">:</span> ./</span><br /><span class="highlight-line"> <span class="token key atrule">ports</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token punctuation">-</span> <span class="token string">'8888:80'</span></span><br /><span class="highlight-line"> <span class="token key atrule">volumes</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token punctuation">-</span> ./<span class="token punctuation">:</span>/var/www/myapp</span><br /><span class="highlight-line"> <span class="token key atrule">links</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token punctuation">-</span> db<span class="token punctuation">:</span>myapp_db</span><br /><span class="highlight-line"> <span class="token key atrule">environment</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token key atrule">TZ</span><span class="token punctuation">:</span> <span class="token string">'America/New_York'</span></span><br /><span class="highlight-line"> <span class="token key atrule">db</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token key atrule">container_name</span><span class="token punctuation">:</span> myapp_db</span><br /><span class="highlight-line"> <span class="token key atrule">restart</span><span class="token punctuation">:</span> always</span><br /><span class="highlight-line"> <span class="token key atrule">image</span><span class="token punctuation">:</span> mariadb<span class="token punctuation">:</span>latest</span><br /><span class="highlight-line"> <span class="token key atrule">ports</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token punctuation">-</span> <span class="token string">'6633:3306'</span></span><br /><span class="highlight-line"> <span class="token key atrule">volumes</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token punctuation">-</span> ./database<span class="token punctuation">:</span>/var/lib/mysql</span><br /><span class="highlight-line"> <span class="token punctuation">-</span> ./res/create<span class="token punctuation">-</span>database.sql<span class="token punctuation">:</span>/docker<span class="token punctuation">-</span>entrypoint<span class="token punctuation">-</span>initdb.d/create<span class="token punctuation">-</span>databse.sql</span><br /><span class="highlight-line"> <span class="token punctuation">-</span> ./res/myapp.sql<span class="token punctuation">:</span>/docker<span class="token punctuation">-</span>entrypoint<span class="token punctuation">-</span>initdb.d/myapp.sql</span><br /><span class="highlight-line"> <span class="token key atrule">environment</span><span class="token punctuation">:</span></span><br /><span class="highlight-line"> <span class="token key atrule">TZ</span><span class="token punctuation">:</span> <span class="token string">'America/New_York'</span></span><br /><span class="highlight-line"> <span class="token key atrule">MYSQL_ROOT_PASSWORD</span><span class="token punctuation">:</span> <span class="token string">'godmode'</span></span></code></pre>
<p>I've highlighted a few of the interesting bits:</p>
<ul>
<li><code>./:/var/www/myapp</code> — mount the current directory into the into the container as a volume at <code>/var/www/myapp</code>, so that code changes I make on my laptop (because that's where I have my editor open) are immediately visible in the container.</li>
<li><code>db:myapp_db</code> — from inside the <strong>web</strong> container, make the <strong>db</strong> container available as hostname <code>myapp_db</code> - Not pictured: My Application.cfc has a datasource configured with the following connectionString: <code>'jdbc:mysql://myapp_db:3306/myapp?...'</code>, so you can see that you simply access the linked docker container by hostname.</li>
<li>mount three volumes for the db container: - <code>./database:/var/lib/mysql</code> uses the local <strong>database</strong> folder as the storage location for all of the data that MariaDB will create. (Starts out empty) - <code>./res/create-database.sql:/docker-entrypoint-initdb.d/create-databse.sql</code> is a script that MariaDB will run on startup if it hasn't already created any databases (appears to be first-run). In this script I create my database and setup the website user account and privileges. After the first time the container is run, when there is data in the <strong>database</strong> folder, this file will be ignored. - <code>./res/myapp.sql:/docker-entrypoint-initdb.d/myapp.sql</code> will run after <code>create-database.sql</code> and is just the most recent production mysqldump prepended with <code>use myapp;</code>. After the first time the container is run, when there is data in the <strong>database</strong> folder, this file will be ignored.</li>
</ul>
<p>With those two problems solved, I moved on to thinking about how to manage all of this. I settled on a Makefile because I knew it would work both on my local Mac and on Ubuntu, where this will ultimately live. (Credit to <a href="http://www.compoundtheory.com/">Mark Mandel</a> for the idea to use Make in the first place. Smart dude.) Here's my Makefile:</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">file_date <span class="token operator">=</span> <span class="token variable"><span class="token variable">$(</span>shell <span class="token function">date</span> +%Y-%m-%d<span class="token variable">)</span></span></span><br /><ins class="highlight-line highlight-line-add"></ins><br /><span class="highlight-line">default:</span><br /><span class="highlight-line"> <span class="token function">docker</span> <span class="token function">ps</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line">db-backup:</span><br /><span class="highlight-line"> <span class="token function">tar</span> czf myapp_db.<span class="token variable"><span class="token variable">$(</span>file_date<span class="token variable">)</span></span>.tar.gz database</span><br /><del class="highlight-line highlight-line-remove"> aws s3 <span class="token function">cp</span> myapp_db.<span class="token variable"><span class="token variable">$(</span>file_date<span class="token variable">)</span></span>.zip s3://myapp/backups/ <span class="token operator">&&</span> <span class="token function">rm</span> <span class="token parameter variable">-f</span> myapp_db.<span class="token variable"><span class="token variable">$(</span>file_date<span class="token variable">)</span></span>.zip</del><br /><del class="highlight-line highlight-line-remove"> <span class="token function">curl</span> https://nosnch.in/my-secret-key</del><br /><del class="highlight-line highlight-line-remove"></del><br /><span class="highlight-line">build:</span><br /><span class="highlight-line"> <span class="token function">docker-compose</span> up <span class="token parameter variable">-d</span> <span class="token parameter variable">--build</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line">up:</span><br /><span class="highlight-line"> <span class="token function">docker-compose</span> up <span class="token parameter variable">-d</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line">bash:</span><br /><span class="highlight-line"> <span class="token function">docker</span> <span class="token builtin class-name">exec</span> <span class="token parameter variable">-it</span> myapp_web /bin/bash</span><br /><span class="highlight-line"></span><br /><span class="highlight-line">logs:</span><br /><span class="highlight-line"> <span class="token function">docker-compose</span> logs <span class="token parameter variable">-f</span></span></code></pre>
<p>Probably the most interesting part of this is the <code>db-backup</code> target. This Makefile is to be used outside of the container, so what this target does is gzip up a copy of the <strong>database</strong> folder and push it into Amazon S3 using their <a href="https://aws.amazon.com/cli/">AWS CLI</a>. Lastly it notifies <a href="http://adamtuttle.codes/the-right-tool-success-notifications/">Dead Man's Snitch</a> that the backup has run so that I can sleep soundly. Should we ever need to restore the database, it should be as simple as unzipping the appropriate backup file into the database folder and starting the container.</p>
<p><img src="https://adamtuttle.codes/img/2016/sit-flying-1.jpg" alt="Nginx welcome screen" /></p>
<h2 id="all-problems-solved.-time-to-go-to-production!" tabindex="-1">All Problems Solved. Time to Go to Production!</h2>
<p>I spent a lot of time over the last month thinking about this. Despite being interested in Google's cloud offerings, I had already sold my client on AWS so I knew I was going to be hosting it there. But should I use their <a href="https://aws.amazon.com/ecs/">EC2 Container Service</a>? If not that, then what? I am aware of Kubernetes but suspect it would be easier to use on Google's cloud than Amazon's. Mostly this whole config was just a big question mark for me.</p>
<p>One thing I did know is that scaling is <em>just not an issue</em>. This app will only ever need one container and that's it. I don't need a cluster, I just need something that will restart it on the off chance the process dies. (That's what <code>restart: always</code> does in docker-compose, by the way.) So while it would be fun to run some sort of clustered setup, it would be the very definition of overkill.</p>
<p>For that reason, I decided to go with a standard vanilla Ubuntu AMI, on which I installed Git, Docker, and the AWS CLI. Because of the agreement I negotiated with my client, I knew I wanted a <code>3-year-reserved-full-upfront</code> EC2 instance, to save money over the on-demand prices. I've used EC2 before, but never reserved instances, so that was another question mark. Like many things AWS, it makes sense once you understand how it works but was not immediately obvious without reading the documentation very carefully.</p>
<h3 id="getting-a-reserved-ec2-instance" tabindex="-1">Getting a Reserved EC2 Instance</h3>
<p>Reserved instances are kind of funny. When you buy it, you're not buying an EC2 instance: You're buying the contract to host an EC2 instance of a certain type, for a certain time period, in a certain location. For example, a <code>t2.large</code> for three years in <code>us-east-1d</code>. The clock starts ticking as soon as the payment form is submitted, so if it were to take a week for you to stand that instance up, you just donated a week's hosting costs to Amazon. Be ready to start that instance before you submit your payment so that you can switch tabs and start it immediately.</p>
<p>The Reserved Instance contracts (I'm just calling them contracts, I don't know if there's a more official nomenclature) are sold on a sort of marketplace. I had heard that if you buy 3 years but end up wanting to terminate early, you're allowed to sell the remainder on the marketplace — I just didn't know how or where to do that. When I was searching for my contract, I only found one option to buy from "3rd party" rather than Amazon.</p>
<p>From there, it works just like any other EC2 instance: save your key file and then connect to the server with SSH: <code>ssh -i ~/.ssh/my_server-root.pem ubuntu@ec2-{my-ip-address}.compute-1.amazonaws.com</code>.</p>
<h2 id="setting-up-my-server" tabindex="-1">Setting up My Server</h2>
<p>The first thing I needed to do was <a href="https://docs.docker.com/engine/installation/linux/ubuntulinux/">install Docker on Ubuntu</a>, which was pretty straight-forward:</p>
<p>All of the below commands are taken from the above Docker on Ubuntu link, after parsing out which commands were necessary for Ubuntu 14.04 Trusty. If you're on a different version, make sure to follow the right docs. And I'm probably running <code>apt-get update</code> more than is strictly necessary, but it was repeated in the guide and I didn't see any harm in running it again, so I just followed instructions. (Standing on the shoulders of giants, remember?)</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">uname</span> <span class="token parameter variable">-r</span><br /><span class="token number">3.13</span>.0-95-generic</code></pre>
<p>Great news! I'm on a recent enough kernel!</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">sudo</span> <span class="token function">apt-get</span> update<br />$ <span class="token function">sudo</span> <span class="token function">apt-get</span> <span class="token function">install</span> apt-transport-https ca-certificates<br />$ <span class="token function">sudo</span> apt-key adv <span class="token parameter variable">--keyserver</span> hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D<br />$ <span class="token function">sudo</span> <span class="token builtin class-name">echo</span> <span class="token string">"deb https://apt.dockerproject.org/repo ubuntu-trusty main"</span> <span class="token operator">></span> /etc/apt/sources.list.d/docker.list<br />$ <span class="token function">sudo</span> <span class="token function">apt-get</span> update<br />$ <span class="token function">apt-cache</span> policy docker-engine<br />$ <span class="token function">sudo</span> <span class="token function">apt-get</span> <span class="token function">install</span> linux-image-extra-<span class="token variable"><span class="token variable">$(</span><span class="token function">uname</span> <span class="token parameter variable">-r</span><span class="token variable">)</span></span> linux-image-extra-virtual<br />$ <span class="token function">sudo</span> <span class="token function">apt-get</span> <span class="token function">install</span> docker-engine<br />$ <span class="token function">sudo</span> <span class="token function">service</span> <span class="token function">docker</span> start<br />$ <span class="token function">sudo</span> <span class="token function">docker</span> run hello-world</code></pre>
<p>Also from that guide:</p>
<blockquote>
<p>For [Ubuntu] 14.10 and below the above installation method automatically configures upstart to start the docker daemon on boot</p>
</blockquote>
<p>... Nice! One less thing to figure out. At this point I was thinking something like, "So the Docker daemon will autostart, but how am I going to make sure my containers autostart, too?" Spoiler alert: I restarted my instance with the containers running and when everything came back up they were brought up too. 👌</p>
<p>At this point I've got Docker running but no docker-compose. Coming from a Mac I was surprised they didn't come bundled together. I was even more surprised that compose wasn't available through apt. Fortunately it was easy enough to google up some instructions on <a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-compose-on-ubuntu-14-04">installing docker-compose on Ubuntu 14.04</a>.</p>
<pre class="language-bash"><code class="language-bash"><span class="highlight-line">$ <span class="token function">sudo</span> <span class="token function">usermod</span> <span class="token parameter variable">-aG</span> <span class="token function">docker</span> <span class="token variable"><span class="token variable">$(</span><span class="token function">whoami</span><span class="token variable">)</span></span></span><br /><ins class="highlight-line highlight-line-add">$ <span class="token function">sudo</span> <span class="token function">apt-get</span> <span class="token parameter variable">-y</span> <span class="token function">install</span> python-pip</ins><br /><span class="highlight-line">$ <span class="token function">sudo</span> pip <span class="token function">install</span> <span class="token function">docker-compose</span></span><br /><span class="highlight-line">$ <span class="token function">which</span> <span class="token function">docker-compose</span> <span class="token operator">&&</span> <span class="token function">docker-compose</span> <span class="token parameter variable">--version</span></span></code></pre>
<p>The <code>usermod</code> line adds the current user (ubuntu) to the <code>docker</code> group, which prevents the need to use sudo every time I want to run a docker command. The last line shows where the compose executable is found and prints its version, just to confirm that all is well.</p>
<p>And now to install the aforementioned AWS CLI:</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">sudo</span> pip <span class="token function">install</span> awscli</code></pre>
<h2 id="configuring-my-app" tabindex="-1">Configuring My App</h2>
<p>Now that all of the pre-requisites are satisfied, it's time to get my containers running and verify that my app is working. I cloned my git repo to <code>/opt/myapp</code> and then <code>chown -R ubuntu ubuntu /opt/myapp</code> so that sudo isn't required for simple things like getting the latest code with <code>git pull</code>.</p>
<p>I have both my <code>Dockerfile</code> for building my web container and my <code>docker-compose.yml</code> in the root of my repo, so it was as simple as <code>cd /opt/myapp && docker-compose up</code> to start them.</p>
<p>Once the containers were done building, I confirmed that the web container was serving the website on port 8888. Internal to the container it's listening on port 80, but that port is being shared on the host as 8888, as you can see in my <code>docker-compose.yml</code> config above. From the host I make an HTTP request to localhost:8888:</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">wget</span> localhost:8888 <span class="token parameter variable">-O</span> /dev/null<br />Resolving localhost <span class="token punctuation">(</span>localhost<span class="token punctuation">)</span><span class="token punctuation">..</span>. <span class="token number">127.0</span>.0.1<br />Connecting to localhost <span class="token punctuation">(</span>localhost<span class="token punctuation">)</span><span class="token operator">|</span><span class="token number">127.0</span>.0.1<span class="token operator">|</span>:8888<span class="token punctuation">..</span>. connected.<br />HTTP request sent, awaiting response<span class="token punctuation">..</span>. <span class="token number">200</span> OK<br />Length: <span class="token number">19789</span> <span class="token punctuation">(</span>19K<span class="token punctuation">)</span> <span class="token punctuation">[</span>text/html<span class="token punctuation">]</span><br />Saving to: ‘/dev/null’<br /><br /><span class="token number">100</span>%<span class="token punctuation">[</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">==</span><span class="token operator">></span><span class="token punctuation">]</span> <span class="token number">19,789</span> --.-K/s <span class="token keyword">in</span> 0s<br /><br /><span class="token number">2016</span>-09-05 09:56:39 <span class="token punctuation">(</span><span class="token number">173</span> MB/s<span class="token punctuation">)</span> - ‘/dev/null’ saved <span class="token punctuation">[</span><span class="token number">19789</span>/19789<span class="token punctuation">]</span></code></pre>
<p>Awesome, it's working!</p>
<h3 id="reverse-proxy" tabindex="-1">Reverse Proxy</h3>
<p>This part isn't strictly necessary. If I only wanted to run this one thing on the server I could just directly expose the container's port 80 as the server's port 80 and call it a day. But putting a reverse-proxy in the middle allows me to add more containers to this server to do other things, which I hope to do in the near future. So I quickly wrapped it up in an <a href="https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-14-04-lts">Nginx</a> reverse proxy:</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">sudo</span> <span class="token function">apt-get</span> <span class="token function">install</span> nginx</code></pre>
<p>Once installed, Nginx is already running:</p>
<p><img src="https://adamtuttle.codes/img/2016/nginx-welcome.png" alt="Nginx welcome screen" /></p>
<p>I didn't spend a lot of time searching out the best Nginx Reverse Proxy tutorial; it was something that was pretty easy to figure out from just the hints in the <a href="https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-as-a-web-server-and-reverse-proxy-for-apache-on-one-ubuntu-14-04-droplet">2nd google result</a> (jump down to "Step 7").</p>
<p>I deleted the symlink at <code>/etc/nginx/sites-enabled/default</code> to disable the welcome screen, then added my own <code>/etc/nginx/sites-available/myapp.com</code> file with this content:</p>
<pre class="language-nginx"><code class="language-nginx"><span class="token directive"><span class="token keyword">server</span></span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">listen</span> <span class="token number">80</span> default_server</span><span class="token punctuation">;</span><br /><br /> <span class="token directive"><span class="token keyword">server_name</span> myapp.com www.myapp.com aws.myapp.com</span><span class="token punctuation">;</span><br /><br /> <span class="token directive"><span class="token keyword">location</span> /</span> <span class="token punctuation">{</span><br /> <span class="token directive"><span class="token keyword">proxy_pass</span> http://127.0.0.1:8888</span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_set_header</span> X-Real-IP <span class="token variable">$remote_addr</span></span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_set_header</span> Host <span class="token variable">$host</span></span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_set_header</span> X-Forwarded-For <span class="token variable">$proxy_add_x_forwarded_for</span></span><span class="token punctuation">;</span><br /> <span class="token directive"><span class="token keyword">proxy_set_header</span> X-Forwarded-Proto <span class="token variable">$scheme</span></span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>Then symlink this file into <code>/etc/nginx/sites-enabled/</code>, and restart nginx. This will proxy all requests to my container that's listening on port 8888, which is good enough for now!</p>
<h2 id="remaining-problems" tabindex="-1">Remaining Problems</h2>
<p>Something's wonky with the system time inside the docker container. By default it comes up as UTC. My host was also UTC. I live in US/Eastern and my client is in US/Eastern so it just makes things easier for my servers to use US/Eastern, too. (Though really, <a href="https://youtu.be/-5wpm-gesOY">Timezones can die in a fire for all I care</a>.)</p>
<p>But I noticed this in the Lucee admin overview at around 22:33 (10:33pm) on Sept 4th:</p>
<p><img src="https://adamtuttle.codes/img/2016/docker-lucee-time.png" alt="Nginx welcome screen" /></p>
<p>What's odd is that it's ahead by ten hours (The container and the host were both in UTC at the time). I changed both to use Timezone <code>America/New_York</code> and restarted both the host and the containers, but that doesn't seem to have helped. Setting the Timezone in the MariaDB container seems to have worked: <code>select current_timestamp;</code> returns my current local time, so I have to imagine there's some disconnect between the system time and Lucee.</p>
<p>Just to verify I did add a comment to a view where I spit out the current value of <code>#now()#</code> and it matches what I see in the Lucee dashboard. So the website thinks that the time is off by a few hours. Not ideal, but I can live with it while I work with the folks in #lucee and #docker to find a fix.</p>
<p>I initially made some scripting changes to my Dockerfile to set the timezone and install NTP, but later found some advice to set an environment variable named <code>TZ</code> with the desired Timezone. That worked too (in that <code>$ date</code> reported the desired date & time), so I ripped out the Dockerfile changes. The environment variable approach works for the MariaDB container too, an added bonus.</p>
<h2 id="conclusion" tabindex="-1">Conclusion</h2>
<p>Soup to nuts, it went surprisingly smoothly. Aside from the Timezone/system time issue, I finished all of that in a day. I started at about 9:00am, took short breaks for lunch and dinner, and finished before 10:00pm. I had mentally prepared myself to stay up until the wee hours of the morning pulling out what's left of my hair, as I usually would in a similar situation, but it just wasn't necessary. The time that I saved, I turned around and used to write this blog post. 🤓</p>
<p><a href="https://twitter.com/AdamTuttle/status/772607180721750018">https://twitter.com/AdamTuttle/status/772607180721750018</a></p>
I Think Donald Trump Is an Idiot, but That Doesn't Mean I Think You Are2016-07-07T12:15:00Zhttps://adamtuttle.codes/blog/2016/trump-is-an-idiot-but-you-arent/<p><img src="https://adamtuttle.codes/img/2016/gettyimages-610603778_wide-5ce473dc30a3b0e4e0d819f5fff536a28c18ae17-s1600-c85.webp" alt="Donald Trump is a moron" /></p>
<p>There are a few moments in my life where I feel like I have truly grown as a person. Usually when I learn something profound and it sinks in immediately, and it's like a lightbulb has lit up over my head. I can think of at least two such moments that happened when I re-watched The West Wing early this year.</p>
<p>The first of which, and the one I want to talk about today, happened as I watched Episode 7 of Season 2, "The Portland Trip." I've embedded a random copy of it that I found online below. You <em>could</em> watch the entire episode, but if you want to jump straight to the part that I'm thinking of, it starts at about the 27 minute mark.</p>
<p>(The West Wing is available on Netflix and I can't recommend it enough, regardless of your personal politics. You can also <a href="https://www.youtube.com/watch?v=cI9HhNjXLu4">rent it on YouTube</a>.)</p>
<p>And here's a transcript from <a href="http://www.westwingtranscripts.com/search.php?flag=getTranscript&id=29">WestWingTranscripts.com</a>:</p>
<blockquote>
<p><strong>SKINNER:</strong>
Ask me the question.</p>
<p><strong>JOSH:</strong>
He compared homosexuality to kleptomania and sex addition, Matt.</p>
<p><strong>SKINNER:</strong>
Yes.</p>
<p><strong>JOSH:</strong>
The Majority Leader. The leader of your own party.</p>
<p><strong>SKINNER:</strong>
He was wrong and I told him so.</p>
<p><strong>JOSH:</strong>
For cryin' out loud!</p>
<p><strong>SKINNER:</strong>
Ask me the question, Josh!</p>
<p><strong>JOSH:</strong>
How can you be a member of this party?!?</p>
<p><strong>SKINNER:</strong>
You've been holding that in for way too long, man.</p>
<p><strong>JOSH:</strong>
This party who says that who you are is against the law.</p>
<p><strong>SKINNER:</strong>
You know, I never understand why you gun control people don't all join the
N.R.A. They've
got two million members. You bring three million to the next meeting... call
a vote...
All those in favor of tossing guns - [Snaps fingers] - Bam! Move on.</p>
<p><strong>JOSH:</strong>
That's a heck of a strategy, Matt. I'll bring that up in a meeting.</p>
<p>Josh sighs heavily as he collapses into his chair.</p>
<p><strong>SKINNER:</strong>
I agree with 95% of the Republican platform. I believe in local government. I'm
in favor
of individual rights rather than group rights. I believe free markets lead
to free people
and that the country needs a strong national defense. My life doesn't have
to be about
being a homosexual. It doesn't have to be entirely about that.</p>
<p>Josh looks at Skinner thoughtfully before deciding not to continue.</p>
<p><strong>JOSH:</strong>
Thanks for coming by.</p>
</blockquote>
<p>With the benefit of hindsight I can say that on many occasions in my life I've thought about specific people as the embodiment of their position on a divisive issue: abortion, gun control, religious freedom, and yes, sexual equality.</p>
<p>When you are hyper focused on a single issue, especially when thinking about someone else, it becomes easy to forget about everything other than that issue; just as Josh Lyman did in the scene transcribed above. But we are all people. Human beings. With an infinite number of thoughts and emotions and desires and needs all struggling to find a place in our priorities. None of us does a perfect job of balancing those out; we all just do the best that we can with the information we've got at the time.</p>
<p>I look at Donald Trump and see a privileged, racist, xenophobic, small-minded, and short-sighted individual whom I am terrified still has a chance of holding our Executive Office. At face value it can be mind blowing.</p>
<p>But you (and I mean that to be directed at his supporters) are just people. You have your own priorities based on your own wants and needs and emotions. I mean, maybe you're an open white supremacist and you like him because he shares your views on immigration. Or maybe you truly believe that Hillary Clinton is going to do something that you view as evil and voting Trump is your way of fighting back. It's also possible that you strongly believe in small government and free markets but you're tired of career politicians who turn their back on you the minute they're sworn into office. These are all legitimate reasons to support Trump, and it is your right to do so.</p>
<p>It would be easy to alias "Trump Supporter" to "Racist Xenophobic Idiot" in my brain, and I admit I have once or twice had those thoughts when seeing some of the coverage of his campaign and supporters. But that is not being honest to what I know to be true.</p>
<p>We're all just people, and we aren't defined by who we support in an election.</p>
TIL: Adding an SSL Cert to the JVM Inside a Docker Image2016-06-27T00:00:00Zhttps://adamtuttle.codes/blog/2016/TIL-adding-a-jvm-ssl-cert-docker/<p>I've been playing with Docker lately for a bunch of reasons, but topping that list is that it's everything I ever wanted out of Vagrant, and then some. (And what better thing to do on a camping trip after the kids are tucked into bed?) In some ways, the Dockerfile syntax is what I wish the Vagrantfile syntax was (shell script instead of Ruby). Anyway, here's a solution to an interesting problem I ran into today.</p>
<p>I need to make HTTP requests to the <a href="https://mailgun.com/app/dashboard">mailgun</a> API and their certificate isn't valid given the JVM cert store that I'm inheriting in my Docker image. Of course, I just need to <a href="http://stackoverflow.com/a/36427118/751">download and import the certificate</a><sup>[1]</sup>, but it took me a little while to wrap my mind around the idiomatic Docker way of doing this.</p>
<p>After some hints from <a href="http://ryanguill.com/">Ryan</a> (to whom I owe cases and cases of beer for similarly helpful advice), and a little more googling and trial and error, this is what I came up with. In my Dockerfile, I've added:</p>
<pre class="language-docker"><code class="language-docker"><span class="token instruction"><span class="token keyword">COPY</span> ./res/mailgun.net.crt /opt/lucee/</span><br /><span class="token instruction"><span class="token keyword">ENV</span> LUCEE_CACERTS /opt/lucee/server/lucee-server/context/security/cacerts</span><br /><span class="token instruction"><span class="token keyword">RUN</span> keytool -noprompt -storepass password_here -import -alias mailgun -keystore <span class="token variable">${LUCEE_CACERTS}</span> -file /opt/lucee/mailgun.net.crt</span></code></pre>
<p>After copying my downloaded mailgun certificate file into the Docker image, I use the <code>keytool</code> command with options <code>-noprompt -storepass</code> to import it without prompting me for the store password.</p>
<p>As a refresher, to find all <code>cacerts</code> files on your (linux) machine, try this:</p>
<pre class="language-bash"><code class="language-bash">$ <span class="token function">find</span> / <span class="token parameter variable">-iname</span> <span class="token string">'cacerts'</span></code></pre>
<p>This will list all files named cacerts on the system. In my Docker image, there were 4. I picked the one associated with <a href="http://lucee.org/">Lucee</a> because that's the app server I'm working with in this case, but it's possible that importing to the JVM's store could have worked too. I just got lucky on my first guess, so I stopped there.</p>
<hr />
<small>
1: On OSX I was able to download the certificate with Firefox following{' '}
<a href="http://stackoverflow.com/a/36427118/751">these instructions</a>, and
while I did select the "der" export type, it saved as a .crt file. Java did
not complain about importing this file.
</small>
Cobbler's Children Syndrome2016-05-05T00:00:00Zhttps://adamtuttle.codes/blog/2016/cobblers-children-syndrome/<p>My old blog at <a href="http://fusiongrokker.com/">fusiongrokker.com</a> was recently offline for more than 230 consecutive hours. Partly, the hosting company is to blame for this. I pay them for a service: A server, connected to the internet, with electricity; and this service was not being properly provided. But partly it is my own fault too.</p>
<p>230 hours is nine and a half days. At what point should I have decided to give up on them coming back online (after my numerous support tickets and phone calls went unanswered) and restored my website onto a new web host from my most recent backups? Well, before you can answer that question it's important to note that I didn't have backups. At least nothing recent.</p>
<p>Oops.</p>
<p>Fortunately, the server is back online now. Yes, I immediately made a backup of both the database and the files for the website. Fortunately, it's not changing in any meaningful way. Maybe a stray comment or two on old articles. Nothing I'll be overly worried about losing. And I intend to get it all converted over to this new blog eventually, with proper redirection from the old URLs. Eventually.</p>
<p>Eventually. That's the word that got me into trouble in the first place. Cobbler's Children Syndrome.</p>
<blockquote>
<p>The cobbler's children have no shoes</p>
</blockquote>
<p>This is a <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/TheCobblersChildrenHaveNoShoes">common trope</a> that most people have probably heard of. On the off chance that you're one of today's <a href="https://xkcd.com/1053/">lucky 10,000</a>, it goes something like this: The cobbler, who makes shoes for a living —for some combination of reasons— doesn't provide shoes for her own children.</p>
<p>And it's not that big of a deal, most of the time. You can walk around most areas barefoot without too much trouble, aside from stores that won't let you in without shoes. But the problem with this mentality is that it fails you when you're desperate. When you find yourself facing a road litered with shards of glass and no choice but to cross it. You'll be cursing your former self for not investing in some shoes.</p>
<p>So it was with my blog's backups. Of course I had backups of <em>important</em> stuff. Things that <em>make me money</em>, like <a href="http://www.restassuredbook.com/">my book</a> and websites I've made for freelance clients. At face value, my personal blog didn't sound so important. <em>I'll get to it eventually</em>, I'd think to myself.</p>
<p>Eight years later... that didn't work out so well for me.</p>
<p>I have no idea how much google-juice those old blog entries lost in the 9.5+ days they were offline, but it was definitely non-zero. I had friends tell me that they couldn't even find my blog in google search results if they were specifically searching for it. Crap.</p>
<iframe src="https://giphy.com/embed/GcSqyYa2aF8dy" width="480" height="358" frameBorder="0" class="giphy-embed" allowFullScreen=""></iframe>
<p>So now it's a priority. Which is to say that it's been prioritized, somewhere near, but not at the top of, my list. I do have to keep doing things that pay the bills first. But after those, this is uh, one of, my top priorities.</p>
<p>There are a lot of things that we as software developers are prone to let slide like this. Version controlling tiny little side projects is another example. And there's only one appropriate response when addressing these questions in your head. There's really no other answer you can give.</p>
<p>Just do it.</p>
The Right Tool for the Job: Success Notifications2016-03-14T09:30:00Zhttps://adamtuttle.codes/blog/2016/the-right-tool-success-notifications/<p>In what I hope is the first in a long series of short posts describing small steps I'm taking to <em>do the right thing™</em>, today I want to talk about eliminating <em>yet another email notification</em>. In a way I guess you could say I'm on a campaign against email, at least as a method of anything other than async remote communication.</p>
<p>Back in the dark ages of 2013 I gave a conference presentation on an open source tool that I use called <a href="http://www.bugloghq.com/">BugLogHQ</a>, an alternative to sending yourself emails containing debug information when an exception occurs in your app – and I hasten to reiterate: one with many advantages. My team and I still use BLHQ to this day, and it is remarkably better than emailing error debug information. <a href="https://adamtuttle.codes/errors-are-best-when-emailed-said-nobody-ever/">You can see the conference session abstract and the slides here</a>. Not as good as being there, but it should give you an idea of how great BLHQ can be, comparatively.</p>
<p>Having solved the error logging problem, the next thing that shows up in my email occasionally, ripe for disruption, is <em>success notifications</em> for scheduled jobs like database backups, and membership expiration reminders. What these types of notifications need is not a success notification, but a failure notification. Hopefully error logging catches any errors that occur during those jobs, but what happens if the cron job gets disabled?</p>
<p><em>Will you notice that you didn't get that email last night when the backups should have run?</em></p>
<p>What you really need in this situation is a <a href="https://en.wikipedia.org/wiki/Dead_man%27s_switch">dead man's switch</a>. These take many forms, but at their simplest you can think of them as a grenade with the pin removed. If you loosen your grip and the "spoon" is released from the grenade ... boom. In digital form, releasing the spoon would be your scheduled job missing a "check in," and the boom would be a notification that your job hasn't run on schedule.</p>
<p><img src="https://adamtuttle.codes/img/2016/brain_grenade.jpg" alt="Grenade" />
<small>Photo credit: <a href="https://www.flickr.com/photos/lapolab/16833901255/">lapolab</a></small></p>
<p>Enter <a href="https://deadmanssnitch.com/r/228ab4a26f">Dead Man's Snitch</a>. This app actually started out as a side project for a local Philly entrepreneur and aquiantance of mine; and I like to support local businesses, especially when they're awesome. DMS is a web service that your job calls to "check in" via a simple HTTP GET request. Miss a check in and they'll let you know. There are a bunch of different schedules available, and at varying price levels you get access to different integrations (i.e. Slack).</p>
<p>There are a few nuances that I'm leaving out to keep this simple, so if you're interested or just have questions, go <a href="https://deadmanssnitch.com/r/228ab4a26f">check out the site</a> for more information. And hey, maybe give the service a shot: you get 1 snitch for free into perpetuity, so why not?</p>
<blockquote><small><em>Disclosure: The links in this post are referral links. If you sign up for an account then I get a free snitch for my account. Obviously I think the service is pretty great: I have a paid account for personal use as well as a paid account for my business; and I'm hoping you will sign up too. But if referral links just aren't your thing, <a href="https://deadmanssnitch.com/">here's a clean one</a>.</em></small></blockquote>
Welcome to OSX, 2016 Edition2016-01-08T13:30:00Zhttps://adamtuttle.codes/blog/2016/welcome-to-osx-2016-edition/<p>Recently a friend told me that he had switched to a Mac full time and asked if I had any advice for software, or just in terms of general usage. I would have swarn that I already had a blog post on the topic that I wanted to link to him (outdated though it may be), but try as I might I couldn't find it. So here we go, I'm starting from scratch!</p>
<h2 id="fundamental-improvements%3A-totalfinder%2C-iterm2%2C-and-totalspaces" tabindex="-1">Fundamental Improvements: TotalFinder, iTerm2, and TotalSpaces</h2>
<p>I find the default experience with Finder to be grating. In my Windows days I used <a href="http://www.ghisler.com/">TotalCommander</a> almost exclusively to deal with the File System, and I had memorized tons of keyboard shortcuts to the point where I almost never needed the mouse. There exists a similar app for OSX, called <a href="http://www.cocoatech.com/pathfinder/">Path Finder</a>, <em>which I don't use.</em> I first stumbled on Path Finder before I was really able to afford spending a bunch of money on apps like that (I bought my first Mac, a base-model 12" macbook, refurbished, on Black Friday). In the interim I started using a free app that made my terminal into a Visor (more on that next), and then later the same developer introduced <a href="http://totalfinder.binaryage.com/">TotalFinder</a>.</p>
<p>I can't say enough good things about TotalFinder: Almost everything that gets on my nerves about the default Finder melts away with TF. I use it as a Visor (pops up from the bottom of my screen when I hit a global keyboard shortcut), frequently use the "dual mode" of side-by-side finders, love the Chrome tabs, and I'm <em>so</em> happy to be able to force folders to sort to the top. There are many more features, but even just those alone are worth the $9 license to me.</p>
<p>Now back to that Terminal visor. BinaryAge, developers of TotalFinder, also had a free product called <a href="http://totalterminal.binaryage.com/">TotalTerminal</a> (no longer actively developed and not compatible with OSX 10.11+) which, for me, harkened back to my days playing the original Quake. With my global keyboard shortcut <code>ctrl+`</code>, an always-available terminal window would slide down from the top of the screen to await my commands. Very handy. For reasons surrounding Zsh and Oh-my-zsh, I've abandoned the default OSX terminal app in favor of <a href="https://www.iterm2.com/">iTerm2</a>, which is much more customizable and also supports the Visor feature.</p>
<p>I have <code>alt+`</code> configured as my Finder visor shortcut, and <code>ctrl+`</code> for my terminal. That's the "back-tick" or "grave" character – the one above my TAB key and what you get if you omit the Shift key when typing <code>~</code>. These two things are core to the way I work on my laptop, and being on a machine without them slows me down and trips me up significantly.</p>
<p>I also use <a href="http://totalspaces.binaryage.com/">TotalSpaces</a>, though I flip between phases where I prefer everything to have its own space and everything in one space with just <code>cmd+tab</code> to switch between them. Currently the latter. Regardless, TotalSpaces is worth $9 to get back the awesome Exposé functionality from the days before Lion. <a href="https://adamtuttle.codes/blog/2011/What-I-HATE-about-OSX-Lion-s-Mission-Control/">I really hate Mission Control</a>.</p>
<h2 id="developer-stuff" tabindex="-1">Developer Stuff</h2>
<p>In my terminal, I prefer to use Zsh with <a href="http://ohmyz.sh/">Oh-my-zsh</a>. There are dozens of great plugins and even more themes to choose from.</p>
<p>I write my code in <a href="http://www.sublimetext.com/3">Sublime Text 3</a>, and also keep my eye on <a href="https://atom.io/">Atom</a> though the plugin ecosystem is still too lacking to make it my daily driver. I have already documented <a href="https://adamtuttle.codes/blog/2013/My-Sublime-Keymap-Common-KB-Shortcuts/">my sublime keymap</a> pretty thoroughly. Plugins, all available through package control:</p>
<ul>
<li>AdvancedNewFile</li>
<li>AutoFileName</li>
<li>AutoSpell</li>
<li>Babel</li>
<li>Babel snippets</li>
<li>ColdFusion (sigh)</li>
<li>Color Highlighter</li>
<li>GitGutter</li>
<li>GutterColor</li>
<li>Indent XML</li>
<li>JavaScriptNext - ES6 Syntax</li>
<li>JSCS-Formatter</li>
<li>JSHint Gutter</li>
<li>LESS</li>
<li>Material Theme</li>
<li>Number King</li>
<li>Package Control</li>
<li>React Templates</li>
<li>ReactJS Snippets</li>
<li>SideBarEnhancements</li>
<li>SublimeLinter</li>
<li>SublimeLinter-jscs</li>
</ul>
<p>I also write my blog posts in Sublime using Markdown. It's all <a href="https://github.com/atuttle/blog">hosted on GitHub</a>, actually. For managing MySQL databases (local and remote) I use <a href="http://www.sequelpro.com/">Sequel Pro</a>.</p>
<p>Pretty much everything else that I use near-daily is <a href="https://nodejs.org/">Node.js</a> and node modules. I currently use Grunt pretty heavily to do LESS, Handlebars.js, and Browserify compiling, along with a few other things (sourcemaps, <a href="http://adamtuttle.codes/modern-cache-busting-for-the-platforms-of-yeasteryear/">cache busting</a>, etc); but I see myself moving toward Gulp/Webpack in the future. Just need time to learn them. If you just need to stand up a quick basic static-file web server in a random directory, I like <a href="https://github.com/knpwrs/nws">nws</a>.</p>
<p>Do enough Node stuff and eventually you'll run into native modules that require compiling locally on your system. That, or if you do any iOS development at all (even with PhoneGap), you're going to need XCode. Better to just bite the bullet early and install it / update it as needed through the App Store.</p>
<p>On the off chance that you need an (S)FTP/S3 client, <a href="https://panic.com/transmit/">Transmit</a> is pretty good.</p>
<p><a href="https://itunes.apple.com/us/app/microsoft-remote-desktop/id715768417?mt=12">Microsoft Remote Desktop</a> is actually reasonably good for managing a few windows boxes remotely. And if you need to test something in Internet Explorer, you can get free virtual machines for most flavors from <a href="https://dev.windows.com/en-us/microsoft-edge/tools/vms/mac/">modern.ie</a>. I use <a href="https://www.virtualbox.org/wiki/Downloads">VirtualBox</a> for my VM's.</p>
<p>Most of the time I do my Git work in the terminal, but occasionally I'll want a GUI for block-level staging or history browsing. In those cases I like <a href="https://www.sourcetreeapp.com/">SourceTree</a>.</p>
<p>I use Keynote for presentations where I won't be doing any live code demos, or various web presentation frameworks when I am.</p>
<h2 id="other-great-stuff" tabindex="-1">Other Great Stuff</h2>
<p>You <em>will not find a better password manager</em> than <a href="https://agilebits.com/onepassword">1Password</a>. Fast, secure, and beautiful to boot. Integrates really well with major browsers and has system keyboard shortcuts for quick access. They even have an Android Keyboard that makes accessing your passwords on the go a snap. I bought a family license and forced it on my wife and mom, too. When my kids are old enough to start having passwords for stuff, they'll be forced into it too.</p>
<p>All of the computers in my house backup to <a href="https://www.backblaze.com/">Backblaze</a>.</p>
<p>For email, I've tried a bunch over the years. None are ever as good as straight up webmail. That said, <em>all</em> of my email is through Google Mail. If you have a work Exchange server or something, I can understand why you would want a local native client. I just don't have a recommendation for you.</p>
<p>I have access to somewhere between half a dozen and a dozen calendars, and coordinating them can be a real pain. I really like <a href="https://calendar.sunrise.am/">Sunrise Calendar</a> and was deeply saddened to hear that they were aqui-hired and will be shutting down the public facing options eventually.</p>
<p>For a basic running todo list I use <a href="https://en.todoist.com/">Todoist</a>. I tried premium but didn't feel the added features were worth it to me. For taking notes, I like <a href="http://writeapp.net/mac/">Write</a>. I sync them with DropBox so that they are also available on my iPod and iPad.</p>
<p>I use <a href="https://www.libreoffice.org/">LibreOffice</a> for office documents. Whatever you do, <a href="http://www.theguardian.com/technology/askjack/2015/sep/03/switch-openoffice-libreoffice-or-microsoft-office">don't use OpenOffice</a>!</p>
<p>If I need to record some or all of my screen, I always use QuickTime Player (should come on your Mac). I don't do a ton of video editing yet, so iMovie is still sufficient for my needs. I see <a href="http://www.apple.com/final-cut-pro/">FinalCut</a> in my future, though. Just like Windows, the best video player is <a href="http://www.videolan.org/vlc/index.html">VLC</a> hands down.</p>
<p>Need to figure out what's eating up so much disk space? Try <a href="http://www.derlien.com/">Disk Inventory X</a>.</p>
<p>I don't have any scientific data to back this up, but subjectively I feel like my eyes are <em>way</em> less strained working on the computer late at night with <a href="https://justgetflux.com/">Flux</a>. It reddens (de-blues?) your screen more as more time passes after sunset.</p>
<p>Have you got more great app suggestions? <a href="https://twitter.com/adamtuttle">Let me know what they are!</a></p>
Expose Node.js on an IIS Server by Reverse Proxying With ARR2015-12-18T08:00:00Zhttps://adamtuttle.codes/blog/2015/add-node-to-existing-iis-server/<p>Here's something I've wanted to be able to do for a long time now: I have a Windows server, running IIS and a dozen or so websites on various technologies like PHP, CFML, and <a href="http://asp.net/">ASP.NET</a>. I'd like to add node.js support to that mix without disrupting any of the existing websites.</p>
<p><strong>Last night I finally figured out how!</strong> It was the result of a lot of google searches and trial-and-error, so I'm no expert on the settings I'll be showing here, but it definitely works. <em>Have you got a better method? Please <a href="https://twitter.com/adamtuttle">hit me up on twitter!</a></em></p>
<p>Granted: Windows is not an ideal environment to host node apps, but you can still get a lot done with it, so you might as well know how to use what's at your disposal.</p>
<h2 id="let's-do-it!" tabindex="-1">Let's do it!</h2>
<p>First things first, you need to be running IIS7 or later, and you need to <a href="http://www.iis.net/downloads/microsoft/application-request-routing">install Application Request Routing (ARR)</a>.</p>
<p>Then, you need your server to accept network connections for your desired hostname. Add a website as you normally would, binding the hostname to the desired IP address:</p>
<p><img src="https://adamtuttle.codes/img/2015/iis-node-website.png" alt="Configure the website as you normally would" /></p>
<p>Next, we need to setup a virtual Server Farm (Microsoft's vocabulary, not mine). After you've installed ARR, you should have a new section in the sidebar of IIS labeled <strong>Server Farms</strong>. Right click that and choose <strong>Create Server Farm</strong>.</p>
<p><img src="https://adamtuttle.codes/img/2015/iis-node-create-server-farm.png" alt="Create a new server farm" /></p>
<p>Give your server farm a name – anything will do – I named mine <code>localhost</code>. Make sure to check the <code>online</code> box. Click the <code>Next</code> button, and enter <code>127.0.0.1</code> as the server address. Again, check the <code>online</code> box and enable <code>Advanced settings...</code>. Expand the newly available advanced settings, and change the http port to match the port your node web app will listen to.</p>
<p>If your node app is using express and binds to port 3000 like so: <code>app.listen(3000)</code>, then you'll enter port 3000 for the <code>httpPort</code> row.</p>
<p><img src="https://adamtuttle.codes/img/2015/iis-node-farm-settings.png" alt="Server farm settings" /></p>
<p>Click the <code>Finish</code> button and then <em>decline</em> the offer to add Rewrite Rules automatically (click the <code>No</code> button).</p>
<p>Select your new Farm in the sidebar, and you should be presented with a list of options like this:</p>
<p><img src="https://adamtuttle.codes/img/2015/iis-node-farm-tiles.png" alt="Server farm tiles" /></p>
<p>Open the <code>Caching</code> options and disable disk cache. This is one of those things that I don't know is strictly necessary for my goal of running Node, but I saw it recommended in a blog post somewhere (regarding <a href="http://asp.net/">ASP.NET</a> sites using Linq), and it's working for me, so why not?</p>
<p>Then go into <code>Routing Rules</code> and select <code>URL Rewrite...</code> from the actions panel on the right. After you do this, you should notice that the Server object in the left panel has become selected, rather than your server farm. This is expected: the change you're making will apply to the whole server.</p>
<p>You shouldn't have any rules configured yet, but if you do, edit it – otherwise create one. Use these settings:</p>
<ul>
<li>Change <code>Regular Expressions</code> to <code>Wildcards</code></li>
<li>Enter <code>*</code> for the pattern</li>
<li>Add a new condition
<ul>
<li>Condition input: <code>{HTTP_HOST}</code> (I had trouble with this at first and it <em>may</em> have been because I copy/pasted that string in. I recommend typing it manually. There should be autocomplete to help.)</li>
<li>Check: <code>Matches the pattern</code></li>
<li>Pattern: The hostname of the website you want to be bound to your node app. For this example I'm using <code>www.hooli.xyz</code>.</li>
</ul>
</li>
<li>Change Action Type to: <code>Route to server farm</code> and select the server farm you created for your node app</li>
</ul>
<p><img src="https://adamtuttle.codes/img/2015/iis-node-rewrite-config.png" alt="Rewrite settings" /></p>
<p>Now start up your node app and load the domain in your browser. It should all be configured and working!</p>
<p>If you want to spin up more node apps, they'll need to listen on a different port, and you'll want to add more Server Farms and routing rules to route requests to a specific farm based on the <code>{HTTP_HOST}</code> value.</p>
<h2 id="where-to-go-from-here" tabindex="-1">Where to go from here</h2>
<p>In theory it should be easy to take this a step further and run multiple node processes for the same app that listen on different ports, and use IIS as a load balancer between them. Perhaps a topic for a future blog entry.</p>
<h2 id="shortfalls" tabindex="-1">Shortfalls</h2>
<p>Coming from a background like CFML or PHP where the app server is (typically) shared by all websites I sometimes found myself wishing for a simpler connection process, but then I would remember that separation of processes is a strength, not a weakness. If one website falls over, none of the others are necessarily affected. Still, it would be nice if it was less work to add a new node-powered website.</p>
<h2 id="what-did-i-get-wrong%3F" tabindex="-1">What did I get wrong?</h2>
<p>I'm probably jinxing it by referencing <a href="https://meta.wikimedia.org/wiki/Cunningham%27s_Law">Cunningham's Law</a>; but here goes anyway. "The best way to get the right answer on the Internet is not to ask a question, it's to post the wrong answer."</p>
<p>Are you dying to point out how horribly wrong I am? <a href="https://twitter.com/adamtuttle">Please tweet at me!</a></p>
My First 100 Jumps2015-12-10T00:00:00Zhttps://adamtuttle.codes/blog/2015/my-first-100-jumps/<p>Just shy of two weeks ago I completed my 100th skydive, which was a huge accomplishment for me. It took me around 18 months to collect that many (not counting the 15 year layoff between jumps #1 and #2).</p>
<p>This isn't some sort of brag: 100 is only a drop in the bucket in the average skydiving career. People that work in the sport can get thousands of jumps in a year. Most active hobbyist jumpers these days don't have much trouble getting 100+ in a year; and in fact I'm hoping to come close to that for calendar year 2016, but for {reasons} I am not able to jump as often as I'd like. Also, there are additional restrictions while you're a student, which I was from May (#2) to August 2014 (#25), resulting in reduced jump numbers.</p>
<p>Let's be honest: I'd jump twenty times every day if I could, so there's probably no reasonable amount that will satisfy the "as much as I'd like" criteria.</p>
<p>Anyway, here is a compilation of some of the video from my first 100 jumps.</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/51kxI9zvauI" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
<p>Jump #100 is not included in the compilation because nobody on the jump had a camera. C'est la vie.</p>
<p>For those of you wondering, in the USA you're not allowed to wear a camera until around 200 jumps. There are more rules and it gets more technical than that, but we'll just leave it at: wearing a camera is something I still have to look forward to, not something I can do yet.</p>
Modern Cache Busting for the Platforms of Yesteryear2015-11-25T00:00:00Zhttps://adamtuttle.codes/blog/2015/modern-cache-busting-for-the-platforms-of-yeasteryear/<p>If you work with a platform born in the last 5-10 years, chances are pretty good you've got something like the <a href="http://guides.rubyonrails.org/asset_pipeline.html">Rails asset pipeline</a> to take care of stuff like this for you. But if your platform of choice (or in my case this time... platform of consequence) doesn't handle it for you, you may be wondering how best to integrate cache busting into your application.</p>
<p>The application I'm working on is built on ColdFusion, and it doesn't do this job for me. There are a lot of things I wish CF did that it doesn't, and when appropriate I solve these problems with a Grunt task. This application has a dozen or so different grunt tasks. Today I added cache busting, and I thought I'd share the process with you. It took me a few minutes of noodling about the best way to <em>automate</em> the process to land on this solution.</p>
<p>Typically, the most basic cache busting is done by appending a query string to the request and changing that query string when you know that the file has changed:</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>link</span> <span class="token attr-name">rel</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>stylesheet<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>style.css?bust_cache=123<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span></code></pre>
<p>I knew I wanted something that didn't require every developer on the team to have to know and remember to update a version number in the layout file any time they made a CSS change. Since we're already using Grunt to compile our LESS into CSS, it made sense to find a way to build it into that process, thus nothing new for the team to remember.</p>
<p>The first step was to register a new Grunt task to create my cache-busting value. I went with an MD5 hash of the current time, but there are lots of other alternative approaches you can employ. Another popular one is to MD5 hash the entire file.</p>
<pre class="language-js"><code class="language-js">grunt<span class="token punctuation">.</span><span class="token function">registerTask</span><span class="token punctuation">(</span><span class="token string">'cache-bust-admin'</span><span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /> <span class="token keyword">var</span> newHash <span class="token operator">=</span> <span class="token function">md5</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">Date</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getTime</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toString</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> fs<span class="token punctuation">.</span><span class="token function">writeFileSync</span><span class="token punctuation">(</span><span class="token string">'./cache-admin.md5'</span><span class="token punctuation">,</span> newHash<span class="token punctuation">,</span> <span class="token string">'utf8'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>With this added to my Gruntfile, I added my new task to the task-set that's run automatically when LESS files change:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token punctuation">,</span><span class="token literal-property property">watch</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">less_admin</span><span class="token operator">:</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token literal-property property">files</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">'www/admin/assets/css/*.*'</span><span class="token punctuation">,</span><span class="token string">'www/admin/assets/less/**'</span><span class="token punctuation">]</span></span><br /><span class="highlight-line"> <span class="token punctuation">,</span><span class="token literal-property property">tasks</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">'css:admin'</span><span class="token punctuation">]</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"><span class="token punctuation">}</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line">grunt<span class="token punctuation">.</span><span class="token function">registerTask</span><span class="token punctuation">(</span><span class="token string">'css:admin'</span><span class="token punctuation">,</span> <span class="token punctuation">[</span><span class="token string">'less:admin'</span><span class="token punctuation">,</span><span class="token string">'cache-bust-admin'</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Now, every time the CSS updates, <code>cache-admin.md5</code> updates too. The next step is to load it into my application, and then reference it from my layout. In my <code>Application.cfc</code> file:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">onApplicationStart</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br /><span class="highlight-line"> <span class="token comment">/* ... */</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line"> application<span class="token punctuation">.</span>cache_hash <span class="token operator">=</span> <span class="token function">fileRead</span><span class="token punctuation">(</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token function">expandPath</span><span class="token punctuation">(</span><span class="token string">'../../cache-admin.md5'</span><span class="token punctuation">)</span><span class="token punctuation">,</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token string">'utf-8'</span></mark><br /><mark class="highlight-line highlight-line-active"> <span class="token punctuation">)</span><span class="token punctuation">;</span></mark><br /><mark class="highlight-line highlight-line-active"><span class="token punctuation">}</span></mark></code></pre>
<p>And the layout:</p>
<pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfoutput</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>link</span> <span class="token attr-name">rel</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>stylesheet<span class="token punctuation">"</span></span> <span class="token attr-name">href</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>style.css?cache=#application.cache_hash#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cfoutput</span><span class="token punctuation">></span></span></code></pre>
<p>Every time we reinitialize our application – part of an automated deploy – it will read the contents of the <code>cache-admin.md5</code> file and store it in memory; then use that value to ensure css is always updated in the browser, while maintaining aggressive cacheing.</p>
Three Reasons We Went Solar (and You Should Too!)2015-11-17T00:00:00Zhttps://adamtuttle.codes/blog/2015/three-reasons-we-went-solar-and-you-should-too/<p>Last June, I called my buddy Mike and told him I wanted to go solar. We were about to move into our new house and I wanted him to look at the roof on Google Maps to confirm it would be worth installing solar panels. At the time he was a salesman (he's since been promoted) for <a href="http://ts.la/adam3818"><s>Solar City</s> Tesla</a>, and I know he'll always be honest with me even when he's trying to sell me something.</p>
<p>Six days ago (in terrible weather for solar energy production) we turned our system on!</p>
<p>Solar energy is really hot right now. Every time I mention our system on Facebook and Twitter I get more people asking me for details about what we paid, how much we're saving, and so on. Everybody wants in. And the good news is it's unbelievably easy, and the savings are real.</p>
<p>Here's our electric consumption compared with our solar production for yesterday. And remember, it's almost winter here in the northern hemisphere, so we're almost opposite of ideal conditions. This is only going to get better during the summer!</p>
<p><img src="https://adamtuttle.codes/img/2015/solar-usage.png" alt="Our solar production vs power consumption for Nov 16th" /></p>
<p>Here, then, are the very first three reasons that come to mind when asked why I wanted to go solar, and you should too:</p>
<h2 id="1%3A-it's-good-for-the-planet!" tabindex="-1">1: It's good for the planet!</h2>
<p>The sun is a renewable energy source that will be around billions of years after the last humans die (if we stay on our current trajectory, at least), and it's free. Just look up. Feel that warmth on your face? Free energy.</p>
<p>Not dumping tons of carbon and other gross polution into the atmosphere is another added bonus.</p>
<p>Really, do I have to sell the whole "it's good for the environment" thing that hard? I think pretty much everyone agrees that solar energy is much better than fossil fuels and easier to harness than wind or water.</p>
<h2 id="2%3A-elon-musk%2C-evil-genius" tabindex="-1">2: Elon Musk, evil genius</h2>
<p>Of course, <s>Solar City</s> Tesla makes some money from the contract they sign with you. How else would they pay for sales staff, installers, and so forth?</p>
<p>The federal government provides a nice Alternative Energy Credit to the owner of the solar system. Since we lease our system from <s>Solar City</s> Tesla, they own it, and they get to pocket that AEC. Then there's the profit from selling any over-produced energy back into the grid. I'll cover some of the details from that in a later section, but for now just know: They make a profit and Elon Musk turns around and reinvests that money in making the world a better place. (That's my educated guess based on his current interests: <s>Solar City</s> Tesla Solar, Tesla Automotive, SpaceX, etc)</p>
<p>I love supporting a business that's out to make the world better, not just to make the shareholders rich.</p>
<h2 id="3%3A-we-save-money-%26-it-costs-us-nothing*" tabindex="-1">3: We save money & it costs us <em>nothing*</em></h2>
<p>I know you're cringing at that asterisk. Give me a chance to explain!</p>
<p>No matter what, there are no up-front costs. We've paid nothing so far, and that has covered a site inspector, the installation crew and the solar panels themselves, and a few inspections after installation. All for $0.</p>
<p>We signed a 20 year, fixed-rate contract at a per-kWh rate that is currently <em>higher</em> than what PECO charges. We're paying 15.67 cents per kWh that our solar panels produce, while PECO is only charging 13.9 per kWh that we consume from the grid. The electric utility industry, on average nationally, increases rates by 5% per year, so by year 6 we'll be saving money compared to PECO, according to those projections.</p>
<p>Our contract is the green bars, and the PECO projection is the red bars.</p>
<p><img src="https://adamtuttle.codes/img/2015/solar-rates.png" alt="Solar City Tesla rates (actual) vs Peco rates (projection) over the next 20 years" /></p>
<p>But what are those blue bars? That's where the asterisk comes in.</p>
<p>We chose a contract that has a higher initial per-kWh rate initially, but is fixed for 20 years. We're going to lose a little bit of money every year until that fixed rate is lower than the PECO rate. But...</p>
<p>For people that want to start saving money immediately, <s>Solar City</s> Tesla offers a similarly-increasing rate plan, but the amount it will increase is fixed at 2.9% and will not change for the life of your contract; <em>and</em> they start you out lower than PECO rates, so you start saving immediately. As you can see, the blue bar starts below the red bar and never goes above it.</p>
<p>How much more are we going to save over the life of the contract by signing up for the fixed rate (green bar) plan?</p>
<p><img src="https://adamtuttle.codes/img/2015/solar-savings.png" alt="Amount saved over time, two Solar City plans compared" /></p>
<p>At the end of your 20 year contract, you've saved a bunch of money... And contributed to saving the planet in a meaningful way! But what's the real dollar difference between the two plans? In order to calculate this we need a rough estimate of the number of kWh used per year, which for my family is currently about 9,200 kWh/y.</p>
<p>On the increasing rate plan (blue bar), we would save about $66 in the first year, and by the end of year 20 we would have saved a combined total of $5,658 and change (about $282/year). On the fixed rate plan (green bar) we're going to lose money for the first few years, and then once our rate is better than PECO's it's going to take a few more years worth of savings to recoup those first years losses; but then the savings take off. We'll lose $229 in our first year, and in year 5 our total combined loss will be up to $637 and change (about $127/year). But by year 20, including those losses up front, we will have saved $7,367 and change compared to the PECO projection (about $368/year).</p>
<p>Because we're willing to take a little bit of a loss for the first few years, we'll save roughly an extra $1,700 over the 20 years of the contract.</p>
<blockquote>
<p>This is probably a good time to mention that your rates <strong>will vary</strong>. These numbers are all based on what <em>my</em> electric utility is charging <em>me</em>, and the rates that <s>Solar City</s> Tesla <em>offered me</em>. Unless you live near me, your starting rates will probably differ, and the rates that <s>Solar City</s> Tesla offers you will probably differ. But the intent will be the same.</p>
</blockquote>
<h2 id="how-can-they-afford-to-do-this%3F" tabindex="-1">How can they afford to do this?</h2>
<p>I don't have special access or Nate-Silver-like abilities to divine truth from seemingly unrelated numbers, but there are a few things that we know for sure.</p>
<ul>
<li><s>Solar City</s> Tesla charges you nothing for your panels and other expenses (installation, etc), and recovers those costs through income streams; making it <em>really easy</em> for anyone with appropriate roof space to get involved. For the average consumer, the cost of the panels alone would be prohibitive to going solar independently.</li>
<li>The federal government (and perhaps some states?) incentivize alternative energy with Alternative Engergy Credits, which is basically a rebate they pay you for using clean energy like solar power. That goes to the owner of the solar system, which is <s>Solar City</s> Tesla, not you.</li>
<li>Your electric utility will pay <s>Solar City</s> Tesla for any energy your system pushes back into the grid (after net metering)</li>
</ul>
<h3 id="what's-net-metering%3F" tabindex="-1">What's net metering?</h3>
<p>When you use electricity from the grid, your power meter spins in one direction. If you produce more energy than you consume, it will be pushed back into the grid and your meter will spin <em>backwards</em>.</p>
<p>Now imagine someone reading your meter on the 1st of the month, and it's at 5,000. When she comes back to read it on the 1st of the following month, it's at 5,300, and you're charged for 300 kWh of usage. But what the meter doesn't show is that during the beginning of the month the weather was outstanding and your meter spun backwards to 4,800; before the sky turned black and you had to start drawing energy from the grid again.</p>
<p>All that matters is that your <em>net</em> usage from the grid was 300 kWh. That's net-metering.</p>
<p>If the reading on the 1st day of the second month was 4,500, that would represent 500 kWh pushed back into the grid, and <em>they</em> would pay <em>you</em> rather than the other way around.</p>
<h2 id="sounds-great%2C-where-do-i-sign-up%3F!" tabindex="-1">Sounds great, where do I sign up?!</h2>
<p>If you're in a state where they operate, <a href="http://ts.la/adam3818">get in touch with them!</a> Or, if you'd rather start talking directly with a person (and I know you), get in touch with me and I'll put you in contact with my salesman Mike.</p>
<h3 id="referral-disclosure" tabindex="-1">Referral Disclosure</h3>
<p>I wrote this post because I couldn't keep up with people asking me for details on my solar setup. I believe in the company (I signed a 20 year contract with them!) and I believe they will do right by you, too. I wouldn't share this information if that weren't true. But, in the interest of disclosure, I do stand to make a little bit of money if you sign up.</p>
<p>If you use <a href="http://ts.la/adam3818">my referral link</a> to initiate contact with <s>Solar City</s> Tesla and you ultimately do get a solar system installed, you and I both get $250.</p>
<p>I hope you do, for our planet's sake.</p>
Extra Life 2015 Recap2015-11-12T08:30:00Zhttps://adamtuttle.codes/blog/2015/extra-life-2015-recap/<p><em>Thank you.</em> Wow.</p>
<p>I really needed to open with that, because I did not deserve the support I received over the course of the 24 hours leading up to my Extra Life marathon—or during it, for that matter—and I am so grateful.</p>
<p>The truth is that I procrastinated too much this year. Not just for Extra Life, but that too. A long time ago I signed up and made my own donation, and when the occasional opportunity arose I asked for donations there in lieu of people offering me beer money for something. But that was it until the day before I was scheduled to sit down for 24 hours of fun and, mostly, isolation. (A rare treat as a parent!)</p>
<p>On Friday evening, 24 hours before go-time, I had amassed $130 in donations. For sure, nothing to be ashamed of; but paling in comparison to my 2013 total of $1,088. The difference, obviously, was that I hadn't made much of an effort at all.</p>
<p>So I wrote a blog post here <a href="https://adamtuttle.codes/extra-life-2015/">soliciting donations</a>, and started hyping it on Twitter and Facebook. I figured if I was going to abandon my family for 24 hours to play games with the excuse that it was for charity, I at least had to <em>try</em> to raise more money. I also planned to stay in a Google Hangout for as much of the marathon as I could. Some people have fancy twitch tv streaming setups (I do not), so pointing my laptop at the tv and my tablet at myself is about the best I can do at the moment.</p>
<p><a href="https://twitter.com/AdamTuttle/status/663117733807034368">https://twitter.com/AdamTuttle/status/663117733807034368</a></p>
<p><a href="https://twitter.com/AdamTuttle/status/663122737343238145">https://twitter.com/AdamTuttle/status/663122737343238145</a></p>
<p>And to my amazement, it happened. I didn't deserve it. I hadn't earned it. But it happened anyway.</p>
<p>My campaign added another $350 from 8 donors over that 48 hour period, bringing my total to $480 – an amount that allowed me to continue making eye contact with Megan when I ask her to tend to every need of our children (and cook for me!) for 24 hours so that I can go play games.</p>
<p>It bears repeating: <em>thank you.</em></p>
<p><img src="https://adamtuttle.codes/img/2015/the-cake-is-a-lie.png" alt="Me after beating Portal (an Extra Life tradition of mine) around 2am" /></p>
<p>And play games I did.</p>
<h2 id="learn-from-my-mistake" tabindex="-1">Learn from my mistake</h2>
<p>This year I wanted to try something different. Usually I would start around 7:00 or 8:00 in the morning, finishing at the same time the following day, and then proceed to spend half (or more) of the day after trying to repay some of my sleep debt. Instead I had the bright idea that I could start at 8pm, finish at 8pm, and head to bed right about the same time as my kids; saving Megan an extra half-day of picking up my slack.</p>
<p>And at least in that regard, it was successful. What I did not plan for was my inability to nap, and thus turning my 24 hour gaming marathon into 24 hours of gaming following 14 hours of being awake for a normal Friday. I was <em>Tired</em> with a capital <em>T</em>. Some time between 1:00am and 2:00am I noticed that I was doing the "long blink" thing, and that even leaning forward on my knees wasn't going to keep me awake. So I stood, playing Portal, with headphones, alone, from about 2:00am to about 4:00am. At one point I started to feel like I was getting a second wind, so I sat down, but I quickly realized I was wrong and went back to standing.</p>
<h2 id="what-i-played" tabindex="-1">What I played</h2>
<p>I started the evening playing <a target="_blank" href="https://www.amazon.com/s/ref=as_li_ss_tl?_encoding=UTF8&camp=1789&creative=390957&field-keywords=star%20wars%20force%20unleashed%20ii&linkCode=ur2&rh=n%3A468642%2Ck%3Astar%20wars%20force%20unleashed%20ii&tag=tuttl-20&url=search-alias%3Dvideogames&linkId=FWUI56KFJBDQBRKV">Star Wars: The Force Unleashed II</a>, a game that I picked up used for a few bucks at GameStop on the recommendation of a clerk because I like Star Wars and enjoyed Dragon Age and Borderlands, but that I had yet to play. It was surprisingly fun. The cut scenes were not trying to be overly-artistic, not too long, and not too short. The game progresses from one conflict to the next very fluidly, and the mechanics are fun and, so far, not too repetitive. (I've not finished it yet.) The boss fights are hard, and occasionally fun stuff happens like you fight while <strong>falling out of the sky</strong> (can't imagine why that would interest me...). The only reason I put it down to play something else was that I was getting too worked up! My hands were cramping and my heart was racing.</p>
<p>To relieve some of the stress from Star Wars, I initiated my Extra Life ritual of playing through the original <a target="_blank" href="https://www.amazon.com/s/ref=as_li_ss_tl?_encoding=UTF8&camp=1789&creative=390957&field-keywords=orange%20box&linkCode=ur2&rh=n%3A468642%2Ck%3Aorange%20box&sprefix=orange%20box%2Cvideogames%2C139&tag=tuttl-20&url=search-alias%3Dvideogames&linkId=YQUNEQHFNZRORCRU">Portal</a>. I'm no <a href="https://www.youtube.com/results?q=portal+speedrun">Portal speed-runner</a>, but this turned out to be the longest game of Portal in history, probably. Between the long-blinks I mentioned above, and my sleep-deprived brain simply not being able to process some of the puzzles (despite having beaten them all many times), I think it took me at least 5 hours. But beat it I did!</p>
<p>Then, because I was in the mood, I started playing <a target="_blank" href="https://www.amazon.com/s/ref=as_li_ss_tl?_encoding=UTF8&camp=1789&creative=390957&field-keywords=portal%202&linkCode=ur2&rh=n%3A468642%2Ck%3Aportal%202&sprefix=portal%202%2Cvideogames%2C128&tag=tuttl-20&url=search-alias%3Dvideogames&linkId=MEY2DSX5JJ63EBGA">Portal 2's</a> single player campaign. The storytelling in the second game is off the charts, and the new mechanics are just unbelievably fun. <em>Juuuust</em> as I was getting into the first puzzles of the new mechanics, around 11:00am, my brother and his girlfriend arrived. They had never been to my new house before (We moved over the summer), so I gave them a quick tour of the house and then we switched over to the two-player version of Portal 2. After a brief lunch break, we decided to play some board games with Megan and the kids.</p>
<p>We played a few hands of <a target="_blank" href="https://www.amazon.com/s/ref=as_li_ss_tl?_encoding=UTF8&camp=1789&creative=390957&field-keywords=exploding%20kittens&linkCode=ur2&rh=n%3A165793011%2Ck%3Aexploding%20kittens&sprefix=exploding%20kittens%2Cvideogames%2C127&tag=tuttl-20&url=search-alias%3Dtoys-and-games&linkId=CERZLY2MKZXETOXE">Exploding Kittens</a> because it's silly and fun, and then a few games of <a rel="nofollow" href="https://www.amazon.com/gp/product/B002SQBB3O/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B002SQBB3O&linkCode=as2&tag=tuttl-20&linkId=AUT2ZMYKS5ZFQ25Y">Tsuro</a>, because it's easy enough for even my 4 year old to play on his own while still being fun and interesting for adults. Once the kids were losing interest in board games, we switched over to some 4-way <a target="_blank" href="https://www.amazon.com/s/ref=as_li_ss_tl?_encoding=UTF8&camp=1789&creative=390957&field-keywords=mario%20kart%20wii&linkCode=ur2&sprefix=mario%20kart%20wii%2Ctoys-and-games%2C127&tag=tuttl-20&url=search-alias%3Dvideogames&linkId=7EBAYYB5SK7O6VNE">Mario Kart Wii</a> – always a favorite.</p>
<p><img src="https://adamtuttle.codes/img/2015/tsuro.jpg" alt="Tsuro is a family favorite!" /></p>
<p>From there we went back to play more Portal 2 until dinner, which was breakfast: Waffles, eggs, and bacon. Yum! My brother left after dinner, and I finished out the marathon playing more of the single player campaign for Portal 2.</p>
<h2 id="why-i-play" tabindex="-1">Why I play</h2>
<p>I was recently reminded of a great scene from the awesome TV show "Louis":</p>
<p><img src="https://adamtuttle.codes/img/2015/parenting.jpg" alt="Checking your neighbor's bowl" /></p>
<p>I have lived a very fortunate life, thanks to the efforts of my parents. We started out somewhere around "poor" and through some combination of hard work and luck manged to move a few rungs up that ladder. I'm reminded of this fact every time an Amazon Prime box containing some goody shows up on my front porch. Every time I buy skydiving equipment. Every time I click my garage door opener. Every time we buy fresh produce at the grocery store.</p>
<p>I don't feel overly guilty for enjoying the fruits of my labor, but I am reminded of my sense of duty to help those less fortunate than me.</p>
<p>When I think about whose life I most want to improve if I only have a limited amount of help to give, it's kids. Without a doubt, 100%. That's why I campaign for <a href="http://extra-life.org/">Extra Life</a>, and why I give to <a href="https://childsplaycharity.org/">Child's Play</a> every year. I don't always have much to give, but I can't think of anyone more deserving of my help than children in need. After all, what did they do to deserve their current situation? They were unlucky to be born to parents without health insurance, or to have a heart condition, or cancer. And with a little helping hand, they have their whole life ahead of them.</p>
<p>I don't have some close personal tie to this cause or these particular charities beyond what I've already said. Neither myself nor any of my family have—to the best of my knowledge—directly benefited from any of these programs. I just believe that they are worthy causes.</p>
<h2 id="it's-not-too-late-to-give!" tabindex="-1">It's not too late to give!</h2>
<p>While my 24 hour gaming marathon is over, the time to donate is not! If you wanted to make a donation but never got around to it, you'll be happy to know that donations are probably still accepted... Unless you're reading this from the future.</p>
<p>If that "Make a donation!" button above still works, then you can still help make a difference in the life of a sick or injured child. And if it's not, there's a good chance that another year of Extra Life is going on now, or will be starting soon. <a href="https://twitter.com/adamtuttle">Reach out to me</a> and I'll be happy to direct you to the best place to make your donation.</p>
<h2 id="next-year..." tabindex="-1">Next year...</h2>
<p>Next year I <em>will not fail to plan.</em></p>
<img src="https://ir-na.amazon-adsystem.com/e/ir?t=tuttl-20&l=ur2&o=1" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
Soliciting Extra-Life Donations for Childrens Hospital of Philadelphia!2015-11-04T08:30:00Zhttps://adamtuttle.codes/blog/2015/extra-life-2015/<p>I know this is terribly last minute, but I've been swamped with deadlines and such and haven't had any time to actively hype my Extra Life campaign this year. <em>And it's happening this weekend!</em></p>
<p>Starting at 8pm on Saturday, November 7th (US-Eastern) I'll begin a 24 hour marathon of games of all types. Video games, Board games, card games, dice games. Maybe even some other tabletop gaming if I can arrange some friends to get together in time. And I won't stop playing, except to eat or go to the bathroom, for 24 hours. No time-outs. No tag-ins.</p>
<iframe src="https://www.extra-life.org/index.cfm?fuseaction=widgets.300x250thermo&participantID=148892" width="302" height="252" frameborder="0" scrolling="no" style="margin: 0 auto 3rem">
<a href="https://www.extra-life.org/index.cfm?fuseaction=donorDrive.participant&participantID=148892">
Make a Donation!
</a>
</iframe>
<p>For the uninitiated, <a href="http://www.extra-life.org/">Extra Life</a> is a charity that collects money for Children's Miracle Network Hospitals, whose mission is to provide care for children, regardless of their family's ability to pay for it.</p>
<p>Kids with cancer, or heart issues, or that were in car accidents. The list goes on and on and on.</p>
<p><img src="https://adamtuttle.codes/img/2015/dylan-xbox.jpg" alt="My son Dylan playing xbox with me" /></p>
<p>They deserve our support and our help! And they're in my back yard. I play for the Children's Hospital of Philadelphia (CHOP), but there are CMN Hospitals all over the world. There's probably one near you too.</p>
<h2 id="what-i'm-doing-for-your-money" tabindex="-1">What I'm doing for your money</h2>
<p>How does it work? Well, I usually compare it to the ACS Relay For Life, except instead of walking in circles for 24 hours we're playing games — of all varieties. The header image is of my brothers and I (and my son) playing some Rock Band during my 2013 marathon. Video games are a big part of it for most people, but I usually try to get my whole family involved by playing board and card games too.</p>
<p><img src="https://adamtuttle.codes/img/2015/robot-turtles.jpg" alt="Robot Turtles, playing with my kids!" /></p>
<p>Just like Relay for Life, it all hinges on soliciting donations from friends. <em><strong>Friends like you</strong>.</em> Would you be so kind as to donate a few dollars to help sick kids? I'm attempting to raise $1,024 this year, and I could really use your help. I greatly appreciate every donation, no matter how small; but more importantly, it goes directly to helping sick kids get the medical care that they need.</p>
<!--
<iframe
src="https://www.extra-life.org/index.cfm?fuseaction=widgets.300x250thermo&participantID=148892"
width="302"
height="252"
frameborder="0"
scrolling="no"
style="margin: 0 auto 3rem"
>
<a href="https://www.extra-life.org/index.cfm?fuseaction=donorDrive.participant&participantID=148892">
Make a Donation!
</a>
</iframe>
-->
<p>What do you say? Let's help some kids!</p>
<!--
<iframe
width="560"
height="315"
src="https://www.youtube.com/embed/ZS7WRl7N1Ig"
frameborder="0"
allowfullscreen
/>
<iframe
width="560"
height="315"
src="https://www.youtube.com/embed/2_sLlWO_v24"
frameborder="0"
allowfullscreen
/>
-->
301 Moved Permanently2015-11-01T22:33:49Zhttps://adamtuttle.codes/blog/2015/301-Moved-Permanently/<p>I've been itching to move my blog for a while now, and it's finally done!</p>
<ul>
<li>New domain name that's unrelated to CFML: <a href="http://adamtuttle.codes/">http://adamtuttle.codes</a></li>
<li>Static site generation with Jekyll</li>
<li>Posts written in markdown</li>
<li>Stored on GitHub: <a href="https://github.com/atuttle/blog">https://github.com/atuttle/blog</a></li>
</ul>
<p>For the time being, my email address will stay the same, for those of you that might have it and be wondering what to do about that. I've been using it for <a href="https://adamtuttle.codes/blog">8+ years</a>, which means I've accumulated a <em>crap-ton</em> of associated website accounts. I couldn't stop using it any time soon, even if I wanted to.</p>
<p>I do want to make an attempt to port all –or most– of my old articles over, at some point. And once that's done, I may make an attempt to import old comments into Disqus — after enabling it for new comments at the new site, of course.</p>
<p>I know that Disqus is not universally loved. I hear it's disabled by some corporate firewalls, which is a shame. But static sites really are pretty awesome, so it's (kind of) a small price to pay.</p>
<p>You can find the new blog here:</p>
<h3 align="center"><a href="http://adamtuttle.codes/">AdamTuttle.codes</a></h3>
<p>Go check out the new site. There's not a lot there, but I'm very happy with it so far, and I have lots of plans to continue improving it.</p>
On Old Dogs and New Tricks2015-10-31T23:59:59Zhttps://adamtuttle.codes/blog/2015/on-old-dogs-and-new-tricks/<p>Consider the following code. This isn't an exercise in figuring out what something does, or how to do it, so I'll just tell you: it updates a list of items (e.g. user input) so that all items in the list are quoted with single-quotes. If it finds them, it strips out double quotes, too (only if both open and close quotes exist).</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line">x <span class="token operator">=</span> <span class="token string">"foo,'bar',"</span><span class="token string">"baz"</span><span class="token string">",qux"</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"></span><br /><span class="highlight-line">x <span class="token operator">=</span> x</span><br /><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">listToArray</span><span class="token punctuation">(</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token keyword">function</span><span class="token punctuation">(</span><span class="token parameter">item</span><span class="token punctuation">)</span><span class="token punctuation">{</span></span><br /><mark class="highlight-line highlight-line-active"> <span class="token keyword">if</span> <span class="token punctuation">(</span>item<span class="token punctuation">.</span><span class="token function">left</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"'"</span> <span class="token operator">&&</span> item<span class="token punctuation">.</span><span class="token function">right</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"'"</span><span class="token punctuation">)</span> <span class="token keyword">return</span> item<span class="token punctuation">;</span></mark><br /><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>item<span class="token punctuation">.</span><span class="token function">left</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">'"'</span> <span class="token operator">&&</span> item<span class="token punctuation">.</span><span class="token function">right</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">'"'</span><span class="token punctuation">)</span><span class="token punctuation">{</span></span><br /><span class="highlight-line"> item <span class="token operator">=</span> item<span class="token punctuation">.</span><span class="token function">left</span><span class="token punctuation">(</span> item<span class="token punctuation">.</span><span class="token function">len</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token operator">-</span><span class="token number">1</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//chop trailing "</span></span><br /><span class="highlight-line"> item <span class="token operator">=</span> item<span class="token punctuation">.</span><span class="token function">right</span><span class="token punctuation">(</span> item<span class="token punctuation">.</span><span class="token function">len</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token operator">-</span><span class="token number">1</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//chop leading "</span></span><br /><span class="highlight-line"> <span class="token keyword">return</span> <span class="token string">"'#item#'"</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span></span><br /><span class="highlight-line"> <span class="token keyword">return</span> <span class="token string">"'#item#'"</span><span class="token punctuation">;</span></span><br /><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span><br /><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">toList</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Would it surprise you to find out that this code is CMFL?</p>
<p>A few short years ago (a lifetime, on the internet), I would have been shocked to see this and learn that it is, indeed, ColdFusion code. It's very <a href="https://en.wikipedia.org/wiki/ECMAScript">ECMAScripty</a>, it's functional, and it uses some of the language trimmings modern modern languages love: map and reduce. <code>array.toList()</code> is really just a reducer, if you think about it.</p>
<p>I'm not heralding ColdFusion, here. I've been working with the platform for more than a decade now and I'm pretty sick of it, to be honest. As a former ColdFusion Product Manager once told me, "You don't really know a programming language unless you can name 10 things you hate about it." By that measure, I'd estimate that I know CFML really, <em>really</em>, <em><strong>really</strong></em> well.</p>
<p>Whatever your opinion of ColdFusion, it is undeniably changing. The next version is on the horizon. They're in private beta (in which I declined to participate); and I would wager public beta is closer than most people would think.</p>
<p><em>Everything old is new again.</em></p>
<p>Me, too.</p>
<p>I'll be 34 in April. While technically not even half way through my career (assuming I retire at 65), 34 feels kind of "old" in tech. I've gone from being one of the hungry young guys that's eager to replace the stagnant developers that came before me, to one of those old kodgers that's unsettled (even if just a little bit) by new JS frameworks coming out every week.</p>
<p>For the last few years my developer career has started to stagnate. I decided not to speak at any conferences in 2015; in part to give myself a break from the crazy amount of work that I put into a presentation that I give only once or twice, but also in part to find out if I would miss it. (I do.) I haven't learned anything new — nothing big, at least. I haven't grown.</p>
<p>I learned the fundamentals of Node.js at the end of 2014, and have been using them to my great advantage since, but that was the last time I truly learned something new. I've been heads-down working on <a href="http://www.alumniq.com/">AlumnIQ</a>, doing the occasional freelance gig, and somehow this has filled my time. As an ambitious person, it seems weird to me to have been able to go an entire year without any major changes (never mind the fact that I bought a new house and moved in the middle of the summer, and I'm a Cub Scout leader now). In some ways I was sating my need for growth by solving hard problems with boring tech.</p>
<p>It makes for (very) brief periods of excitement—problem solved! hooray! 🎉🍻—interspersed with lots of headaches and hair pulling. By contrast, when learning a new piece of technology, every time you master another bit of it you're rewarded with endorphins for having learned something new and exciting.</p>
<p>The first step is admitting you have a problem.</p>
<p>In the last two weeks, I've started learning <a href="https://facebook.github.io/react/">React</a> and really making an effort to learn <a href="https://babeljs.io/docs/learn-es2015/">ES2015</a> (I've only dabbled until now). I haven't combined the two yet. I don't yet understand ES2015 well enough to be able to look at it and understand how it works instantly, like I can with ES5. I'm also very interested in the static-website-generator craze. In fact I'm writing this blog post as my first new entry to a <a href="https://jekyllrb.com/">Jekyll</a> blog that I intend to host via <a href="https://pages.github.com/">GitHub Pages</a>.</p>
<p>2016 will be a high-growth year for me. That's my goal. I guess I'm a little early for the New Years Resolution party, but planning ahead is a good way to set myself up for success, right?</p>
<p>Now if you'll excuse me, I have some React components to write.</p>
Taffy 3.1.0-RC12015-09-28T13:22:52Zhttps://adamtuttle.codes/blog/2015/Taffy-3-1-0-RC1/<p>Yep, you read that right! It's time to start testing <a href="https://github.com/atuttle/Taffy/archive/v3.1.0-RC1.zip">Taffy 3.1.0-RC1</a> in your environments. I've been using the bleeding-edge code in production for months now without issue. All that was holding up the release was a few minor tickets and some documentation that needed to be written, and over the weekend (and some early this morning) I <a href="https://github.com/atuttle/Taffy/milestones">knocked almost all of that out</a>.</p>
<p><img src="https://adamtuttle.codes/img/2015/taffy-3.1.0-milestone.png" alt="Milestoneriffic" /></p>
<p>The only remaining ticket is to <a href="https://github.com/atuttle/Taffy/issues/270">find a good place to document the X-HTTP-METHOD-OVERRIDE header</a>. (Any ideas?)</p>
<p>As you can read for yourself in the <a href="http://docs.taffy.io/3.1.0/#What-s-new-in-3-1-0">Release Notes</a>, this release includes 11 bug fixes and 13 enhancements. The most noticeable changes all center around the dashboard. In particular, one that's been bugging me for a long time: It was getting too cluttered with stuff that you probably didn't need for a majority of your API's / requests. Now all of those extra non-essential features are hidden and easily toggled from a set of links:</p>
<p><img src="https://adamtuttle.codes/img/2015/taffy-3.1.0-dashboard.png" alt="Milestoneriffic" /></p>
<p>Please <a href="https://github.com/atuttle/Taffy/archive/v3.1.0-RC1.zip">Download Taffy 3.1.0-RC1</a> and give it a shot with your API. If you have any issues, please <a href="https://github.com/atuttle/Taffy/issues">file a ticket</a> or we can talk about it in the <strong>#taffy</strong> channel on the <a href="http://cfml-slack.herokuapp.com/">CFML slack</a> team.</p>
REST Web APIs: The Book2014-12-04T08:45:00Zhttps://adamtuttle.codes/blog/2014/REST-Web-APIs-The-Book/<script type="text/javascript" src="https://gumroad.com/js/gumroad.js"></script>
<p><img src="https://adamtuttle.codes/img/2014/REST-Web-APIs-The-Book.png" width="100" hspace="3" style="max-width:100px;border:2px solid #d0d0d0;border-radius: 3px;" />I've decided to write a book about designing REST Web APIs. I'm calling it <a href="https://gumroad.com/l/rest-the-book"><strong>REST Web APIs: The Book</strong></a>.</p>
<p align="center"><a href="https://gum.co/rest-the-book" class="btn btn-lg btn-warning" style="text-decoration: none"><i class="fa fa-book"></i> Pre-Order REST Web APIs for $12</a></p>
<p>I get asked for my opinion on some facet of API design at least once a week, so I decided that I would try to distill my opinions into a concise, pragmatic book so that you can take me with you wherever you go. On a thumb drive or your kindle! (It's going to be an ebook, at least at first.)</p>
<p>This is <em><strong>not</strong></em> a ColdFusion or Taffy book. It <em>will</em> have Taffy (and thus ColdFusion) code toward the end to illustrate the concepts described throughout the book. It's not entirely conceptual, but the bits that are concrete will be referring to the browser or raw http requests moreso than the code you write to make your API work.</p>
<p>I'm targeting completion on or by Noon, December 19th. Of this year. Two weeks from now. Yes, I know that's a little bit crazy; but I'm attempting to embrace the "Just Ship It!" mantra. I'm sure there will be updates, but <em>they will always be free for every buyer</em>, so what are you waiting for?</p>
<p><a href="https://gumroad.com/l/rest-the-book/backer">Go pre-order my book!</a></p>
<p>Once it's done I'll be charging $19 for it. If you pre-order, you can have it for $12, as a thank you for believing in me. <s>And if you're one of <a href="https://gumroad.com/l/rest-the-book/backer">the first five buyers to use this link you get even more of a discount</a> for your enthusiasm.</s> Sold Out!</p>
<p>I'll be teasing more information over the next two weeks, but for now I'm just going to leave it at that.</p>
<p><s>Did I mention that <a href="https://gumroad.com/l/rest-the-book/backer">the first 5 pre-orders at this link</a> get an extra discount? I did? Good.</s> Sold Out!</p>
<p align="center"><a href="https://gum.co/rest-the-book" class="btn btn-lg btn-warning" style="text-decoration: none"><i class="fa fa-book"></i> Pre-Order for $12</a></p>
Taffy 3.0.02014-11-19T10:28:35Zhttps://adamtuttle.codes/blog/2014/Taffy-3-0-0/<p>We did it! Taffy 3.0.0 is here.</p>
<p><a href="https://github.com/atuttle/Taffy/archive/v3.0.0.zip" class="btn btn-success" style="text-decoration:none">Download Taffy 3.0.0</a></p>
<p>As I mentioned with each Release Candidate, there are a few <a href="http://docs.taffy.io/3.0.0/#Breaking-Changes">Breaking Changes</a> — things you will have to change in your API code if you upgrade Taffy. I've documented what each of them is in the new <a href="http://docs.taffy.io/3.0.0/#Breaking-Changes">Breaking Changes</a> section of the documentation, and not to worry: They're all pretty trivial. Just changing a few base classes and stuff like that.</p>
<p>You can also see a <a href="http://docs.taffy.io/3.0.0/#What-s-new-in-3-0-0">full detailed list of all bug fixes and new features in the documentation</a>.</p>
<p>As ever, my immense thanks go out to everyone that contributed to this release; and there are many of you! I've added your GitHub avatars to the bottom of <a href="http://taffy.io/">taffy.io</a> as a thank you!</p>
Taffy 3.0.0-RC32014-11-13T12:27:56Zhttps://adamtuttle.codes/blog/2014/Taffy-3-0-0-RC3/<p>Today I've published <a href="https://github.com/atuttle/Taffy/releases/tag/v3.0.0-RC3">Taffy 3.0.0-RC3</a>, which I expect to be the final Release Candidate build for the final 3.0.0 release. Just a few small bug fixes went into this update, and unless anyone reports any issues with this build it will become the official 3.0.0 build some time next week.</p>
<p>There have been <em>many</em> changes between 2.x and 3.0, a few of which will require some minor updates to your code, so if you're upgrading please familiarize yourself with the <a href="http://docs.taffy.io/3.0.0/#Breaking-Changes">breaking changes</a> first.</p>
Taffy 3.0.0-RC12014-09-19T14:44:08Zhttps://adamtuttle.codes/blog/2014/Taffy-3-0-0-RC1/<p>Today I created the first Release Candidate for version 3 of Taffy: <a href="https://github.com/atuttle/Taffy/releases/tag/v3.0.0-RC1">Version 3.0.0-RC1</a></p>
<p>Not much has changed since <a href="https://adamtuttle.codes/blog/2014/Taffy-3-0-0-alpha/">the alpha</a>. There were a few bug fixes, and I've gotten a few pull requests that seemed like a good fit for this release, so they have been added as well. All of the additions have been added to the <a href="http://docs.taffy.io/3.0.0/#What-s-new-in-3-0-0">"What's New" section of the documentation</a>.</p>
<p>While this is not officially a "stable" release, it is my estimate that it <em>could be</em>. I'm using it in production without any issues; but I'm also sure my code doesn't make use of every single code-path through the framework. If you have some spare cycles it would be greatly appreciated if you could try upgrading your install and make sure everything still works.</p>
<p>Don't forget that there are <a href="http://docs.taffy.io/3.0.0/#Breaking-Changes">a few things that are going to require some code changes on your part</a>, but they are relatively minor.</p>
<p>If all goes well with the RC, we could be looking at Taffy 3.0.0 stable in as little as a week! As always, thank you for your help testing and for your pull requests.</p>
GitHub Tip for Your First Pull Request2014-09-10T00:00:00Zhttps://adamtuttle.codes/blog/2014/your-first-github-pull-request/<p><img src="https://adamtuttle.codes/img/2014/darinka-kievskaya-ff221Bu56mI-unsplash.jpg" alt="Dog pulling on a rope toy" /></p>
<p class="photo-byline">Photo by <a href="https://unsplash.com/@darisja?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Darinka Kievskaya</a> on <a href="https://unsplash.com/s/photos/pull?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></p>
<p>There is a mistake that I see a lot of people new to GitHub, or new to making Pull Requests, making. It's something I find myself explaining often in people's first PR threads; so I thought I would clean it up a little bit and share it here for the benefit of a wider audience.</p>
<p>In Git, unlike SVN and other centralized Version Control Systems, branches are easy, fast, and awesome. You should be using the heck out of them. And, if you're not, then that can come back to bite you when you get around to sending pull requests. I'll illustrate the problem with an example.</p>
<p>Let's say you want to contribute something to <a href="https://github.com/foundeo/cfdocs">CFDocs</a>. So you fork the repository, clone your fork to your local machine, and edit the code. You're only planning on doing this one little thing, so why bother making a branch?</p>
<p>You make your change, everything looks great, so you open a pull request. You select that you want to merge in the changes from your master branch into the original repo's master branch:</p>
<p><img src="https://adamtuttle.codes/img/2014/pr-1.png" alt="Pull Request Screen Shot" /></p>
<p>You submit your pull request, and happily go on about your business. All is right with the world.</p>
<p>As it happens, it takes <a href="https://foundeo.com/">Pete</a> a few days to get to your Pull Request, but in the meantime, you've become inspired at how easy it was to do that, so you want to help more!</p>
<p>You make more changes, push them up to your repository, and then... oops. Because you still have an open Pull Request against your master branch, pushing new commits to that branch updates the Pull Request and adds those commits. Oops indeed! These new commits have nothing to do with the original PR and really should not be included.</p>
<p>The solution to this problem is <strong>branches</strong>.</p>
<p>If you create a branch for each feature, or for each discrete pull-request that you want to submit, then they will stay in their separate silos. This is particularly important if the project author asks you to make changes (add tests, change formatting, etc). Then you can update the code in one branch without affecting the other; and as luck would have it, these changes are automatically merged into the PR when you push them up to your fork on GitHub.</p>
<p>Here's a more appropriate workflow:</p>
<ol>
<li>Fork the repo to your account</li>
<li>Clone the repo to your local machine: <code>git clone https://github.com/atuttle/cfdocs.git</code> (replace the account and repo names, of course)</li>
<li>Create a branch to do your work in, and name it after what you're working on: <code>git checkout -B ormExecuteQueryImprovements</code></li>
<li>Make your changes, and commit them to your local clone</li>
<li>Push your changes, in their branch, back up to GitHub: <code>git push -u origin ormExecuteQueryImprovements</code></li>
<li>Then, open your PR and select this new branch as the HEAD of the PR.</li>
<li>Profit</li>
</ol>
<p>I hope this saves you the headache of adding commits to the wrong Pull Request and helps you enjoy your experience with Git and GitHub a little bit more.</p>
<h2 id="update" tabindex="-1">Update</h2>
<p>It was suggested that I also add information on pulling in upstream changes to stay up to date. Here you go!</p>
<p>As so often happens after you've forked, coded, and pull-requested, the project continues to move forward — instantly making your clone outdated and out of sync. If you want to continue to contribute, you need to pull in those "upstream" (e.g. from the canonical repo) changes so that your future pull requests are based on the latest code and can be more easily merged. If you don't, there's a chance that your changes will be difficult to merge, and possibly even based on outdated code, making them (hate to say it, but) nearly worthless! For ongoing participation via fork and pull-request, this process is absolutely critical.</p>
<p>Here's what I do:</p>
<ol>
<li>Add a new remote to your local clone of your fork, named upstream, and using the git url (http or ssh) of the original repository that you forked. In my example I'm using the url for the <strong>cfdocs</strong> repo in the <strong>foundeo</strong> organization:<br /><code>git remote add upstream https://github.com/foundeo/cfdocs.git</code></li>
<li>Before starting new changes of your own, make sure you've got the latest code from upstream: <code>git checkout master</code> to get your own master branch and then <code>git pull --rebase upstream master</code> to merge in any changes from upstream.</li>
</ol>
<ul>
<li><strong>update-update:</strong> I've recently decided to start using the <code>--rebase</code> flag on all of my pulls <a href="http://gitready.com/advanced/2009/02/11/pull-with-rebase.html">to keep the revision history cleaner</a>.</li>
</ul>
<ol>
<li>Now create your new branch for your new feature/bug-fix: <code>git checkout -B arrayMapImprovements</code></li>
<li>Make your changes and commit them.</li>
<li>Merge upstream master again, into your branch this time (still just <code>git pull --rebase upstream master</code> — default target is your current branch), just to make sure that anything that changed since you started coding is also already merged (e.g. if it took you several days to make your changes, or if the canonical repo is fast-moving)</li>
<li>Submit your new pull request from your new branch. (Once accepted and merged into the original repository, your local feature branches can be deleted)</li>
<li>Rinse and repeat</li>
</ol>
Taffy 3.0.0-Alpha2014-08-15T09:00:00Zhttps://adamtuttle.codes/blog/2014/Taffy-3-0-0-alpha/<p>The first <em><strong>alpha</strong></em> of Taffy version 3.0.0 is ready for public testing!</p>
<p>There was a recent burst of development, thanks to some corporate sponsorship in the form of this conversation with my employer:</p>
<blockquote>
<p>"Taffy can do that, right?"</p>
<p>"No, but it's something I've wanted to add for a while."</p>
<p>"Go do it."</p>
</blockquote>
<p>There have been a lot of changes and improvements, and one of those is that the documentation now includes a <a href="http://docs.taffy.io/3.0.0/#What-s-new-in-3-0-0">What's new</a> section, so I'm not going to copy over all of that content into the blog post. There are a few <a href="http://docs.taffy.io/3.0.0/#Breaking-Changes">breaking changes</a> from 2.x to 3.0 that you should be aware of, but in terms of how long it will take you to update your code, the impact is only a minute or two of your time.</p>
<p>The test suite is fully updated as well, so please <a href="https://github.com/atuttle/Taffy#running-the-tests">run the tests</a> on your environment and report back if you have any troubles! I've verified things are working well on Adobe ColdFusion 8.0.1 and 10u13, and Railo 4.2 (Cheers to <a href="https://adamtuttle.codes/blog/2014/Taffy-3-0-0-alpha/cfmlblog.adamcameron.me">Adam Cameron</a> for helping out with this!), but I can't check every platform. As always, if you run into any problems, we've got the <a href="https://groups.google.com/forum/#!forum/taffy-users">mailing list</a> and you can <a href="https://github.com/atuttle/Taffy/issues">file bug reports</a>.</p>
<p><strong>Also important:</strong> If you try out v3 and <strong>don't</strong> have any problems, please let me know that, too! Silence either means nobody is testing or nobody is having problems, and there's a world of difference.</p>
<p>As always, thank you to everyone that submits bug reports and feature requests, participates on the mailing list, and talks about Taffy in the #ColdFusion channel on IRC. My biggest thanks this time go out to <a href="https://github.com/danshort">Dan Short</a>, <a href="https://github.com/jbvanzuylen">Jean-Bernard van Zuylen</a>, and <a href="https://github.com/phipps73">phipps73</a> for their pull requests.</p>
<p>If all goes well with the alpha over the next few weeks, I'll release an official v3.0.0 by the end of the month.</p>
Taffy 2.2.0 Watermelon Released2014-01-21T10:00:50Zhttps://adamtuttle.codes/blog/2014/Taffy-2-2-0-Watermelon-Released/<p>The biggest news this time around is that we now have a shiny new logo!*</p>
<p><img src="https://raw.github.com/atuttle/Taffy/master/dashboard/logo-lg.png" alt="Taffy Logo" /></p>
<p>As ever, the <a href="https://github.com/atuttle/Taffy/releases/tag/v2.2.0">full release notes for Taffy 2.2.0</a> are available on GitHub.</p>
<p><img src="https://adamtuttle.codes/img/2014/taffy_metrics.png" alt="Taffy Metrics" /></p>
<p>In addition to the new logo, my second favorite addition is that Taffy now returns additional headers that show you where time was spent on the server, to help find any performance bottlenecks that may pop up. These values are listed in milliseconds and are not perfect because there is some rounding involved.</p>
<ul>
<li><strong>TIME-IN-ONTAFFYREQUEST</strong> should be obvious, this is the time it takes for your OTR function to run.</li>
<li><strong>TIME-IN-PARSE</strong> is the time needed to figure out what's being requested. This can get somewhat complex at times. I would expect the value to go up if you're using a bunch of tokens, for example.</li>
<li><strong>TIME-IN-RESOURCE</strong> is the amount of time spent inside your resource cfc, for example, running the GET method.</li>
<li><strong>TIME-IN-SERIALIZE</strong> is the amount of time needed to serialize the data to the requested format. Handled by your custom representation class or the framework default -- either way it will be reported here.</li>
<li><strong>TIME-IN-TAFFY</strong> is the end timestamp minus the start timestamp of the request, minus the other duration values above. It's conceivable but unlikely that this might go negative! Again, that would just be a rounding error -- no big deal.</li>
</ul>
<p>I'm so glad that <a href="https://groups.google.com/forum/#!topic/taffy-users/OSXdem_tfeU">Richard Poole requested this feature</a> because while I always knew that Taffy is <em>ridiculously fast</em> I didn't have any numbers to back that up. Now I do, and Taffy is even faster than I expected!</p>
<p>Thanks to <a href="https://groups.google.com/forum/#!topic/taffy-users/OSXdem_tfeU">Richard Poole</a>, <a href="https://github.com/gensior">Jesse Franceschini</a>, <a href="https://github.com/codymartin">Cody Martin</a>, <a href="https://github.com/marbetschar">marbetschar</a>, <a href="https://github.com/leeahoward">Lee Howard</a>, <a href="https://github.com/finalcut">Bill Rawlinson</a>, <a href="https://github.com/AaronMartone">Aaron Martone</a>, <a href="https://github.com/cfchef">Tony Junkes</a>, <a href="https://github.com/commadelimited">Andy Matthews</a>, and <a href="https://github.com/w1nterl0ng">w1nterl0ng</a> for your bug reports and other contributions to this release!</p>
<p>* Stroz can <em>"occupy me"</em> if he doesn't like it.</p>
Announcing Taffy.io2013-12-05T08:45:00Zhttps://adamtuttle.codes/blog/2013/Announcing-Taffy-io/<p>One of the weakest points in Taffy, for, well, forever, has been the (organization of the) documentation. To be fair, it's not <em>entirely</em> my fault! GitHub wiki's kind of suck a little bit.</p>
<p>Fortunately I've taken steps to improve the information architecture.</p>
<h2 id="step-1%3A-get-an-easy-to-remember-domain" tabindex="-1">Step 1: Get an easy to remember domain</h2>
<ul>
<li><a href="http://taffy.io/">taffy.io</a> is now the primary landing page</li>
<li><a href="http://docs.taffy.io/">docs.taffy.io</a> is the new home of the documentation!</li>
</ul>
<h2 id="step-2%3A-cleanly-separate-docs-for-different-versions" tabindex="-1">Step 2: Cleanly separate docs for different versions</h2>
<p>I know the documentation doesn't have the prettiest formatting, but that will get better with time. One of the key features of this architecture is that the documentation for each version of Taffy is now available separately (one of my biggest gripes with GH Wikis).</p>
<p><img src="https://adamtuttle.codes/img/2013/Taffy_2.1.0_Documentation.png" alt="Taffy Documentation Version Selector" /></p>
<p>I've created a docs set for Taffy 2.1.0, but I have yet to update it with the changes from 2.0.1 to 2.1.0 (coming soon-ish!)</p>
<h2 id="step-3%3A-enable-simple-collaboration" tabindex="-1">Step 3: Enable simple collaboration</h2>
<p>Another one of my gripes with GH Wikis is that they are either globally editable or completely locked down. I <a href="https://github.com/gollum/gollum/issues/265">voiced my desire for pull requests for wikis long ago</a>, but that will probably never see the light of day. That also contributed to the decision to <a href="https://github.com/atuttle/TaffyDocs">house the docs in their own repo</a>. As a result, anyone can now submit a pull request via normal channels for the documentation.</p>
<h2 id="step-4%3A-automate-all-the-things!" tabindex="-1">Step 4: Automate all the things!</h2>
<p>Fortunately, this change opened up opportunities for much cool integration. The docs are now being "continuously deployed" -- that is, whenever a pull request is merged or I push a change to GitHub, the documentation source is automatically compiled to HTML and deployed to <a href="http://docs.taffy.io/">docs.taffy.io</a>. Cool stuff, eh?</p>
Taffy 2.1.0 Released2013-11-25T09:00:00Zhttps://adamtuttle.codes/blog/2013/Taffy-2-1-0-Released/<p>Extree, extree, read all about it!</p>
<p>Taffy 2.1 is <a href="https://github.com/atuttle/Taffy/releases/tag/v2.1.0">here</a>! Another small, semi-frequent release. Since Taffy 2.0 in August, we've (yes, we, not I!) have fixed 5 bugs -- none too big, looks like 2.0 was a really stable release! -- and added 6 new features. As is the new custom, you can read <a href="https://github.com/atuttle/Taffy/releases/tag/v2.1.0">the full release notes on GitHub</a>.</p>
<p>The new feature I'm most excited about is that <a href="https://github.com/atuttle/Taffy/issues/155#issuecomment-27259068">method metadata is now passed to onTaffyRequest</a>. This enables neat stuff like adding role-based security as resource-function metadata:</p>
<h2 id="example-resource%3A" tabindex="-1">Example Resource:</h2>
<pre class="language-js"><code class="language-js">component <span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.resource"</span> taffy_uri<span class="token operator">=</span><span class="token string">"/foo"</span> <span class="token punctuation">{</span><br /> <span class="token keyword">function</span> <span class="token function">get</span><span class="token punctuation">(</span><span class="token punctuation">)</span> role<span class="token operator">=</span><span class="token string">"read-data"</span> <span class="token punctuation">{</span><br /> <span class="token comment">//...</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<h2 id="application.cfc%3A" tabindex="-1">Application.cfc:</h2>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">onTaffyRequest</span><span class="token punctuation">(</span><span class="token parameter">verb<span class="token punctuation">,</span> cfc<span class="token punctuation">,</span> requestArguments<span class="token punctuation">,</span> mimeExt<span class="token punctuation">,</span> headers<span class="token punctuation">,</span> methodMetadata</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /><br /> local<span class="token punctuation">.</span>user <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token operator">...</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">//get user from token or something...</span><br /><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">structKeyExists</span><span class="token punctuation">(</span>methodMetadata<span class="token punctuation">,</span> <span class="token string">"role"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> availableRole <span class="token keyword">in</span> local<span class="token punctuation">.</span>user<span class="token punctuation">.</span>roles<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>availableRole <span class="token operator">==</span> methodMetadata<span class="token punctuation">.</span>role<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><br /> <span class="token punctuation">}</span><br /> <span class="token keyword">return</span> <span class="token function">newRepresentation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">noData</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">withStatus</span><span class="token punctuation">(</span><span class="token number">403</span><span class="token punctuation">,</span> <span class="token string">"Not Authorized"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><span class="token keyword">else</span><span class="token punctuation">{</span><br /> <span class="token comment">//no role required on the method, so allow anyone to use it</span><br /> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>Note the new 6th argument to <code>onTaffyRequest</code>: <code>methodMetadata</code>. You can call it whatever you like; arguments are passed positionally, not by name.</p>
<hr />
<p>This release includes pull requests from <a href="https://github.com/billrawlinson">Bill Rawlinson</a>, <a href="https://github.com/jbvanzuylen">Jean-Bernard van Zuylen</a>, and <a href="https://github.com/dominic-oconnor">Dominic OConnor</a>. Thanks, guys!</p>
<hr />
Taffy 2.0.0, Codename: Gif-Not-Jif2013-08-30T08:45:00Zhttps://adamtuttle.codes/blog/2013/Taffy-2-0-0-Codename-Gif-not-Jif/<p>It only took 3 years, but Taffy 2.0 is finally here. If you attended <a href="https://adamtuttle.codes/blog/2013/Taffy-Office-Hours-Episode-0-Recording/">Office Hours</a> last week, you got a sneak peek at the updated dashboard UI. Now you can use it yourself!</p>
<p image="" lost="" to="" the="" annals="" of="" internet,="" sorry!=""></p>
<p>In the past (e.g. <a href="https://adamtuttle.codes/blog/2013/Taffy-1-3-codename-Excelsior/">Version 1.3</a>) I have detailed the release notes here on my blog; but that was before GitHub added their fancy new "Releases" feature for doing just that. So this time, instead of listing everything here, I'll point you to the official <a href="https://github.com/atuttle/Taffy/releases/tag/v2.0.0">Taffy 2.0 release notes</a>. Please do read them, as there are some significant changes between versions 1.3 and 2.0, as you probably expect.</p>
<p>As discussed when I released version 1.3, I'm aiming for smaller, more frequent builds; on the order of 2 or 3 releases per year. As such, 2.0 is mostly a maintenance release with a new coat of paint. Aside from the visual redesign of the dashboard, the only new feature is JSONP support. It also includes 8 bug fixes, and everything previously deprecated has been removed.</p>
<p>As always, I offer my sincerest thanks to everyone involved with Taffy in any way, shape, or form. Every single bug report, feature request, support request, mailing list thread, tweet and blog post honestly warms my heart.</p>
<p>I still have big plans for improving the framework, the documentation, and the experience of forking and sending a pull request, so stay tuned!</p>
Taffy Bug Warning: Danger when Managing Custom Representation Class in a Bean Factory2013-07-30T08:35:33Zhttps://adamtuttle.codes/blog/2013/Taffy-Bug-Warning-Danger-when-Managing-Custom-Representation-Class-in-a-Bean-Factory/<p>Just a quick note to let anyone out there using Taffy in a certain way know that I found <a href="https://github.com/atuttle/Taffy/issues/147">a pretty severe bug</a> last night.</p>
<ul>
<li>If you're not using a <a href="https://github.com/atuttle/Taffy/wiki/Using-a-Custom-Representation-Class">Custom Representation Class</a> ("CRC"), this bug does not affect you.</li>
<li>If you're not storing said CRC in a Bean Factory, this bug does not apply to you.
<ul>
<li>To do this, you would either put your CRC somewhere inside your <code>/resources</code> folder, or define it in your bean factory config (e.g. coldspring.xml)</li>
</ul>
</li>
</ul>
<p>I have yet to test this using ColdSpring, but the gist of the problem is that instead of using a clean copy of the representation class for every request, the same one is used over and over again (only when using the above-described configuration). This results in race conditions, and even with 100% sequential requests, could result in response data, status code/text, and headers from one response bleeding into later responses.</p>
<p>There are two known workarounds right now. Implementing either one by itself should completely mitigate the problem:</p>
<ul>
<li>Don't store your CRC inside of a Bean Factory; not even Taffy's <code>/resources</code> folder.</li>
</ul>
<p><s>or...</s></p>
<ul>
<li><s>For every response, in every resource, make sure you explicitly define all output:</s>
<ul>
<li><s>Data (It's possible that the <code>noData()</code> method is affected, I haven't tested this either...</s></li>
<li><s>Status Code</s></li>
<li><s>Status Text</s></li>
<li><s>Additional Headers (if you don't need to add extra headers, pass <code>{'{}'}</code>)</s></li>
</ul>
</li>
</ul>
<p><s>Obviously the former approach should be easier to implement on larger API's.</s></p>
<blockquote>
<p>After further consideration, the second workaround proposed above is not a good idea. If a race condition occurs, responses could still bleed into one another.</p>
</blockquote>
<p>For a little bit more information and status updates as I work on the resolution, keep an eye on <a href="https://github.com/atuttle/Taffy/issues/147">this bug report</a>. The fix will go into the Bleeding Edge release soon. Both the 1.3 and 1.2 branches will be updated soon thereafter, so if you're still on a supported version of Taffy you can expect fixes!</p>
<p>Apologies for the bug. If it softens the blow any, I found it while working on a project fast approaching its deadline, so it has pretty greatly inconvenienced me, too!</p>
<p>I'll post here on my blog once the updated versions are available for download.</p>
Advanced Authentication with Taffy APIs2013-07-10T09:04:57Zhttps://adamtuttle.codes/blog/2013/advanced-authentication-with-taffy-apis/<p>I get a lot of advice requests for Taffy implementations, but the one I get most commonly has to be non-trivial authentication.</p>
<p>What separates trivial from non-trivial? What I'd consider "trivial" authentication would be using SSL and requiring HTTP Basic Auth with every request. It's simple and it doesn't reinvent the wheel. In version 1.3, Taffy also added <a href="https://github.com/atuttle/Taffy/wiki/Authentication-and-Security">a helper method for getting the Basic Auth credentials</a> so that you can easily validate them and respond as appropriate.</p>
<p>Non-trivial would be anything more complicated than that. Anything up to and including OAuth. I've never had the pleasure of implementing OAuth, but I regularly do a slightly simplified version, which I'll explain for you in this post.</p>
<p>OAuth was designed for the use-case when there are 3 separate parties involved; for example Twitter, your Twitter App of choice, and you -- you being the granting authority for your Twitter account. OAuth is a way for you to tell Twitter that it's ok for {Twitter App Of The Week} to use your account on your behalf, without giving that application your password, and in such a way that you and Twitter can work together to revoke that app's privileges instantly and at any time.</p>
<p>I have done a lot of APIs where there are only two involved parties: The account holder ("you") and the API/client; because the API provider also provides (codes) the apps. What I'm about to describe works very well, and actually pretty easily, in this last situation.</p>
<h2 id="let's-talk-theory" tabindex="-1">Let's Talk Theory</h2>
<p>My apps generally allow you to use multiple devices -- perhaps you have both an iPad and an iPhone, and you want to use our app from both. Nothing wrong with that, right? But there's always the concern that someone might get your password or steal your phone and start doing less than desirable things with your account.</p>
<p>You, as the API designer, want to be able to revoke a specific device's access individually (e.g. kick the 2nd iPad connected to your account out, without kicking out the 1st iPad or the iPhone), so that means each device will authenticate and then from that point on include a token with each request instead of the username & password. Each device gets a unique token, and all tokens are tied back to a single account. An account can have many device tokens.</p>
<p>As far as the user is concerned, they just need to log in once on each device, and from then on every time they use the app they're already logged in. As far as I -- the APP & API administrator -- am concerned, you or I (on your behalf if I detect suspicious activity) can easily block a single discrete device at any time by revoking its token.</p>
<p>Sound Good? Let's Implement It.</p>
<h2 id="step-1%3A-authentication-%26-token-generation" tabindex="-1">Step 1: Authentication & Token Generation</h2>
<p>The first thing you need is an API endpoint that accepts authentication requests and will return the device its own token on a successful authentication.</p>
<pre class="language-js"><code class="language-js">component<br /><span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.resource"</span><br />taffy_uri<span class="token operator">=</span><span class="token string">"/authenticate"</span> <span class="token punctuation">{</span><br /><br /> <span class="token keyword">public</span> <span class="token keyword">function</span> <span class="token function">post</span><span class="token punctuation">(</span><span class="token parameter">username<span class="token punctuation">,</span> password<span class="token punctuation">,</span> deviceId</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><br /><br /> <span class="token comment">//implementation of verifying the username/password are up to you...</span><br /> <span class="token keyword">var</span> authSuccess <span class="token operator">=</span> <span class="token function">authenticate</span><span class="token punctuation">(</span>username<span class="token punctuation">,</span> password<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>authSuccess<span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token function">noData</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">withStatus</span><span class="token punctuation">(</span><span class="token number">401</span><span class="token punctuation">,</span> <span class="token string">"Not Authorized"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token comment">//create a new device token:</span><br /> <span class="token keyword">var</span> deviceToken <span class="token operator">=</span> <span class="token function">createUUID</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token comment">//saving the device token to the database is up to you...</span><br /> <span class="token function">saveDeviceToken</span><span class="token punctuation">(</span>username<span class="token punctuation">,</span> deviceId<span class="token punctuation">,</span> deviceToken<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token comment">//tell the device auth was successful and give it the device token</span><br /> <span class="token keyword">return</span> <span class="token function">representationOf</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token literal-property property">token</span><span class="token operator">:</span> deviceToken <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">withStatus</span><span class="token punctuation">(</span><span class="token number">200</span><span class="token punctuation">,</span> <span class="token string">"OK"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">authenticate</span><span class="token punctuation">(</span><span class="token parameter">username<span class="token punctuation">,</span> password</span><span class="token punctuation">)</span><span class="token punctuation">{</span><span class="token punctuation">}</span><br /> <span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">saveDeviceToken</span><span class="token punctuation">(</span><span class="token parameter">username<span class="token punctuation">,</span> deviceId<span class="token punctuation">,</span> deviceToken</span><span class="token punctuation">)</span><span class="token punctuation">{</span><span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>You'll notice that in addition to the username and password we're also accepting a deviceId. The majority of my mobile development experience is with Phonegap, wherein we can access the device id on Android, and a UUID that represents the device-installation on iOS (this will change if the user uninstalls and reinstalls the application). You can capture whatever metadata you like here. In addition to deviceId, we tend to also grab OS version and a few other little things so that we can data-mine our install base, as well as tie any in-app feedback back to a particular device (e.g. lots of reports of a bug from iPad2 but nothing from the original iPad). The sky is the limit, you just need to store this metadata with the device token.</p>
<p>If you wanted to, you could add an additional check for an existing device token for the supplied device Id, and return that if it exists instead of creating a new one.</p>
<h2 id="step-2%3A-require-the-token-for-(almost)-all-requests" tabindex="-1">Step 2: Require the Token for (Almost) All Requests</h2>
<p>This is done via the Taffy lifecycle event method <code>onTaffyRequest</code>, in <code>Application.cfc</code>:</p>
<pre class="language-js"><code class="language-js">component <span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.api"</span> <span class="token punctuation">{</span><br /><br /> <span class="token keyword">function</span> <span class="token function">onTaffyRequest</span><span class="token punctuation">(</span><span class="token parameter">verb<span class="token punctuation">,</span> cfc<span class="token punctuation">,</span> requestArguments<span class="token punctuation">,</span> mimeExt<span class="token punctuation">,</span> headers</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /><br /> <span class="token comment">//allow white-listed requests through</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span>cfc <span class="token operator">==</span> <span class="token string">"authenticate"</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token comment">//otherwise require a device token</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span><span class="token function">structKeyExists</span><span class="token punctuation">(</span>requestArguments<span class="token punctuation">,</span> <span class="token string">"deviceToken"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /><br /> <span class="token keyword">return</span> <span class="token function">newRepresentation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">noData</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">withStatus</span><span class="token punctuation">(</span><span class="token number">401</span><span class="token punctuation">,</span> <span class="token string">"Not Authenticated"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token comment">//and make sure it's valid</span><br /> <span class="token punctuation">}</span><span class="token keyword">else</span> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span><span class="token function">validateToken</span><span class="token punctuation">(</span>requestArguments<span class="token punctuation">.</span>deviceToken<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /><br /> <span class="token keyword">return</span> <span class="token function">newRepresentation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">noData</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">withStatus</span><span class="token punctuation">(</span><span class="token number">403</span><span class="token punctuation">,</span> <span class="token string">"Not Authorized"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token punctuation">}</span><br /><br /> <span class="token comment">//if a token is included, and valid, allow the request to continue</span><br /> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">validateToken</span><span class="token punctuation">(</span><span class="token parameter">token</span><span class="token punctuation">)</span><span class="token punctuation">{</span><span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>We verify that the <code>cfc</code> argument is <code>authenticate</code>, because the authentication cfc was named <code>authenticate.cfc</code>. <a href="https://github.com/atuttle/Taffy/wiki/Organizing-your-resources-into-subfolders">BeanId rules apply</a>. This effectively white-lists every method in the <code>authenticate</code> cfc, but if you want to be more selective you could also check the verb argument to see which method is being requested.</p>
<h2 id="where-do-we-go-from-here%3F" tabindex="-1">Where do we go from here?</h2>
<p>We've now setup a situation in which all requests require a device token (a.k.a. API Key), unless currently authenticating. This device token implies a few things, and it's possible to inject them into the request context so that they're made available in every resource automatically. For example, what if most of your resources wanted to know the username of the account making the request? Instead of having the device include it in the request details, you can look it up during request startup and add it to the request arguments.</p>
<p>Here we'll be building onto <code>onTaffyRequest</code>. For the sake of brevity I'm going to <em>omit</em> the parts I wrote above for authentication. Just know that you can (and should) include the code for both. I would do what follows after everything from the previous example, before the final <code>return true;</code>.</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">onTaffyRequest</span><span class="token punctuation">(</span><span class="token parameter">verb<span class="token punctuation">,</span> cfc<span class="token punctuation">,</span> requestArguments<span class="token punctuation">,</span> mimeExt<span class="token punctuation">,</span> headers</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /><br /> <span class="token keyword">var</span> userMetadata <span class="token operator">=</span> <span class="token function">getUserMetadata</span><span class="token punctuation">(</span>requestArguments<span class="token punctuation">.</span>deviceToken<span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">structAppend</span><span class="token punctuation">(</span>requestArguments<span class="token punctuation">,</span> userMetadata<span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /><br /><span class="token punctuation">}</span><br /><br /><span class="token keyword">private</span> <span class="token keyword">function</span> <span class="token function">getUserMetadata</span><span class="token punctuation">(</span><span class="token parameter">deviceToken</span><span class="token punctuation">)</span><span class="token punctuation">{</span><span class="token punctuation">}</span></code></pre>
<p>If <code>getUserMetadata</code> returns a structure, all of the data from the structure will be passed to the requested resource as if it were part of the request. You can use the device token to lookup the username, or userId, or device type, or myriad other things available in your data through that user-association. Be careful, however: This happens for <em>every</em> request. Don't use it lightly. (e.g. Don't run two database queries when one will do.)</p>
<p>How do you access this injected data? The same way you access other request arguments: through method arguments.</p>
<pre class="language-js"><code class="language-js">component<br /><span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.resource"</span><br />taffy_uri<span class="token operator">=</span><span class="token string">"/widgets/sprockets/{sprocketTypeId}"</span> <span class="token punctuation">{</span><br /><br /> <span class="token keyword">function</span> <span class="token function">get</span><span class="token punctuation">(</span><span class="token parameter">sprocketTypeId<span class="token punctuation">,</span> color<span class="token punctuation">,</span> userId</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /><br /> <span class="token comment">// the device will include the sprocketTypeId in the URI, and the desired color</span><br /> <span class="token comment">// in the request arguments but you've used the deviceToken to look up the</span><br /> <span class="token comment">// userId and injected it into the request arguments all three values are</span><br /> <span class="token comment">// available through the method arguments</span><br /><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>I hope this has clearly explained what I consider to be a relatively complex (but no OAuth!) authentication mechanism that is robust and secure. When you break it down into discrete chunks like this, there's really not a whole lot to it, and that's a big part of why I like Taffy's approach. Less is more!</p>
<p>Finally, remember that there are an infinite number of ways to remove the epidermis from your feline. Don't feel like you have to do it this way -- this is just an explanation of how I like to set things up, in the hopes that it inspires you or possibly even completely meets your needs.</p>
Using Taffy Without an Application.cfc Base Class2013-05-17T16:06:12Zhttps://adamtuttle.codes/blog/2013/Using-Taffy-Without-an-Application-cfc-Base-Class/<p>One of the most common complaints I hear about Taffy is that it requires that you extend its base class in your Application.cfc...</p>
<pre class="language-js"><code class="language-js">component <span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.api"</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></code></pre>
<p>Sometimes you don't want to do this. Maybe you have another base class you're using, or you just don't want to. Whatever your reasons, it's a dealbreaker.</p>
<p>Talking with <a href="https://twitter.com/cfchris">Chris Phillips</a> at cf.Objective() today, I had an epiphany that it should be possible to get Taffy running without extending <code>taffy.core.api</code> from your Application.cfc. The only reason it's been a requirement thus far was to automatically bootstrap in Taffy's handling of the <code>onApplicationStart</code>, <code>onRequestStart</code>, and <code>onRequest</code> events.</p>
<p>But if you're willing to wire those up manually, you can write your Application.cfc any way you like as long as -- at some point -- you hand off the request to the appropriate Taffy method. Here's an Application.cfc that uses Taffy as an internal object without using extends in Application.cfc:</p>
<pre class="language-js"><code class="language-js">component <span class="token punctuation">{</span><br /><br /> <span class="token keyword">function</span> <span class="token function">onApplicationStart</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> application<span class="token punctuation">.</span>taffy <span class="token operator">=</span> <span class="token function">createObject</span><span class="token punctuation">(</span><span class="token string">'taffy.core.api'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> application<span class="token punctuation">.</span>taffy<span class="token punctuation">.</span><span class="token function">onApplicationStart</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">function</span> <span class="token function">onRequestStart</span><span class="token punctuation">(</span><span class="token parameter">thePage</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> application<span class="token punctuation">.</span>taffy<span class="token punctuation">.</span><span class="token function">onRequestStart</span><span class="token punctuation">(</span> arguments<span class="token punctuation">.</span>thePage <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /> <span class="token keyword">function</span> <span class="token function">onRequest</span><span class="token punctuation">(</span><span class="token parameter">thePage</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> application<span class="token punctuation">.</span>taffy<span class="token punctuation">.</span><span class="token function">onRequest</span><span class="token punctuation">(</span> arguments<span class="token punctuation">.</span>thePage <span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><br /><span class="token punctuation">}</span></code></pre>
<p>You can put whatever you want before and after those calls that defer to Taffy's internal implementations (e.g. the rest of your application), and it should still work exactly as described in the documentation -- <code>variables.framework</code> would still be used, etc -- but I haven't tested this extensively yet.</p>
<p>Just a little tidbit that I thought some of you may be interested to hear.</p>
Errors Are Best When Emailed... Said Nobody Ever2013-05-17T00:00:00Zhttps://adamtuttle.codes/blog/2013/errors-are-best-when-emailed-said-nobody-ever/<p>Exceptions happen. Bugless code is a myth.</p>
<p>I hope I don't have to convince many people that sending every exception via email is not the greatest idea since sliced bread. "Sure, it sucks... But what else can I do?" would be a natural response. I've been there. How else can you keep track of exactly what's going wrong in your application? How else will you be alerted when something goes wrong?</p>
<p>For many of you, email probably plays the role of courier between your application and your bug database (JIRA, Bugzilla, etc). But that's not what email was intended for, and to be honest, it's a poor tool for the job, so it does a crappy job.</p>
<p>The truth is that there are better options, but nobody can blame you for not knowing about them... Until now. I'm going to show you the two most popular options available for CF, how to use them to get useful analytical data back about your application's health, or lack thereof, and how you can extend the concept beyond just your CFML. And the real surprise is that it's easy.</p>
<iframe src="https://speakerdeck.com/player/6142a1bd3e2b408786cf9c508c953d5c" style="width:100%;height:700px;">
</iframe>My Sublime Keymap & Common KB Shortcuts2013-03-08T08:06:26Zhttps://adamtuttle.codes/blog/2013/My-Sublime-Keymap-Common-KB-Shortcuts/<p>What follows is nothing new or unique; it's just a collection of keyboard shortcuts and tips that I've picked up from various blog posts, videos, and personal recommendations from friends over the last ~year of using Sublime full time. I'll go over each setting on its own here in the post, but if you just want the keybindings, <a href="https://gist.github.com/atuttle/5111814">you can find them in this gist</a>.</p>
<blockquote>
<p><strong>Comma first!</strong> You'll notice that I'm using a comma-first formatting. I like it because it makes missing (or superfluous, in the case of the first line) commas stick out like a sore thumb. You don't have to like it or use it, but it suits me well.</p>
</blockquote>
<h2 id="line-swapping" tabindex="-1">Line swapping</h2>
<pre class="language-json"><code class="language-json"><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"alt+up"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"swap_line_up"</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"alt+down"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"swap_line_down"</span> <span class="token punctuation">}</span></code></pre>
<p>Alt+up / Alt+down to move lines up or down. This is a neat trick I picked up from Eclipse back in the day, and I use it a lot; especially to move large chunks of code -- just highlight several lines and use it.</p>
<h2 id="duplicate-line" tabindex="-1">Duplicate line</h2>
<pre class="language-json"><code class="language-json"><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"ctrl+alt+down"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"duplicate_line"</span> <span class="token punctuation">}</span></code></pre>
<p>Ctrl + Alt + Down creates a copy of the current line on the next line.</p>
<p><strong>Pro-tip:</strong> If you have less than 1 line selected, that text is duplicated and placed after the selection. No more copy+paste to create a quick series of <code><br/></code>...</p>
<h2 id="delete-line" tabindex="-1">Delete line</h2>
<pre class="language-json"><code class="language-json"><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"alt+d"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"run_macro_file"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span><span class="token property">"file"</span><span class="token operator">:</span> <span class="token string">"Packages/Default/Delete Line.sublime-macro"</span><span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre>
<p>Another favorite shortcut from Eclipse was Cmd+D; but Sublime does something <del>else</del> <strong>better</strong> with Cmd+D, so with a little effort I re-trained my brain to use Alt+D instead.</p>
<h2 id="split-view" tabindex="-1">Split View</h2>
<pre class="language-json"><code class="language-json"><span class="token punctuation">,</span><span class="token punctuation">{</span><br /> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"ctrl+alt+left"</span><span class="token punctuation">]</span><br /> <span class="token punctuation">,</span><span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"set_layout"</span><br /> <span class="token punctuation">,</span><span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span><br /> <span class="token property">"cols"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token number">0.0</span><span class="token punctuation">,</span> <span class="token number">0.75</span><span class="token punctuation">,</span> <span class="token number">1.0</span><span class="token punctuation">]</span><br /> <span class="token punctuation">,</span><span class="token property">"rows"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token number">0.0</span><span class="token punctuation">,</span> <span class="token number">1.0</span><span class="token punctuation">]</span><br /> <span class="token punctuation">,</span><span class="token property">"cells"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">]</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span><br /><span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"ctrl+alt+right"</span><span class="token punctuation">]</span><br /> <span class="token punctuation">,</span><span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"set_layout"</span><br /> <span class="token punctuation">,</span><span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span><br /> <span class="token property">"cols"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token number">0.0</span><span class="token punctuation">,</span> <span class="token number">0.25</span><span class="token punctuation">,</span> <span class="token number">1.0</span><span class="token punctuation">]</span><br /> <span class="token punctuation">,</span><span class="token property">"rows"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token number">0.0</span><span class="token punctuation">,</span> <span class="token number">1.0</span><span class="token punctuation">]</span><br /> <span class="token punctuation">,</span><span class="token property">"cells"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">]</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p><em>I don't always use split view, but when I do, it's <strong>F&^%ing useful</strong>.</em></p>
<p>I use the above commands in conjunction with the built-in commands for Super+Alt+1/2/... Using Super+Alt+2 puts you in split-mode with each side taking up 50%, but sometimes you want to focus more on one side or the other. For instance, maybe your CSS is nice and tidy and doesn't need much horizontal space, so you want to give that available space to the HTML on the other side. With the above additions, Ctrl+Alt+Right gives 75% of the code's screen real estate to the right pane and 25% to the left (think "focus right") and Ctrl+Alt+Left ("focus left") gives 75% to the left pane and 25% to the right. Rarely do I ever need 3 panes, but when I do I just resort to the mouse for any resizing that needs to be done.</p>
<p><strong>Pro-tip:</strong> Use Super+K+B to toggle off the file navigator for even more screen real estate during split view.</p>
<p>In addition to the above shortcuts that arrange panes, these move the cursor to the specified pane (e.g. super+1 focuses the left-most pane, super+2 focuses the 2nd pane from the left, etc):</p>
<pre class="language-json"><code class="language-json"><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"super+1"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"focus_group"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"group"</span><span class="token operator">:</span> <span class="token number">0</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"super+2"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"focus_group"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"group"</span><span class="token operator">:</span> <span class="token number">1</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"super+3"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"focus_group"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"group"</span><span class="token operator">:</span> <span class="token number">2</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"super+4"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"focus_group"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"group"</span><span class="token operator">:</span> <span class="token number">3</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre>
<p>And these shortcuts move the focused file to the specified pane:</p>
<pre class="language-json"><code class="language-json"><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"alt+1"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"move_to_group"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"group"</span><span class="token operator">:</span> <span class="token number">0</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"alt+2"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"move_to_group"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"group"</span><span class="token operator">:</span> <span class="token number">1</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"alt+3"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"move_to_group"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"group"</span><span class="token operator">:</span> <span class="token number">2</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"alt+4"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"move_to_group"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"group"</span><span class="token operator">:</span> <span class="token number">3</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre>
<h2 id="bookmarks" tabindex="-1">Bookmarks</h2>
<pre class="language-json"><code class="language-json"><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"f3"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"next_bookmark"</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"shift+f3"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"prev_bookmark"</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"super+f3"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"toggle_bookmark"</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"super+shift+f3"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"clear_bookmarks"</span> <span class="token punctuation">}</span><br /><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"alt+f3"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"select_all_bookmarks"</span> <span class="token punctuation">}</span></code></pre>
<p>Bookmarks are crazy-useful when you find yourself jumping back and forth around a long file. When working on an app with a <em>long</em> Mach-II.xml file I'll often bookmark the section of events that I'm working on, and views, and listeners, so that I can quickly jump back and forth between these sections.</p>
<p>All of these are modifications to the built-in bookmark settings because I use a plugin called <a href="https://github.com/titoBouzout/SideBarEnhancements">Sidebar Enhancements</a> to add in some extra functionality for improving keyboard-focused workflow, and it overrides the default F2 shortcut for flipping between bookmarks into a rename-focused-file action. I spent many years on Windows, so I'm actually quite happy to have this action and F2 is easily remembered because Windows uses F2 for file renames; so I just bumped everything for bookmarks up to F3 instead.</p>
<h2 id="create-xml%2Fhtml-tag-pair" tabindex="-1">Create XML/HTML Tag Pair</h2>
<pre class="language-json"><code class="language-json"><span class="token punctuation">,</span><span class="token punctuation">{</span> <span class="token property">"keys"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"alt+,"</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token property">"command"</span><span class="token operator">:</span> <span class="token string">"insert_snippet"</span><span class="token punctuation">,</span> <span class="token property">"args"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"name"</span><span class="token operator">:</span> <span class="token string">"Packages/XML/long-tag.sublime-snippet"</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre>
<p>This is another re-mapping of built-in functionality. By default the shortcut for this is Ctrl+Shift+W, but I found that awkward to perform, so I switched it to Alt+< (it's really Alt+, but I think of it as Alt+< because I'm creating a tag-pair). This is a very handy tool, because it changes the closing tag to match the opening tag as you type. It defaults to a P, but the "p" portion is highlighted and if you overwrite that to "div" then the closing tag automatically becomes <code></div></code>. Another great thing about this is that when you hit space and start adding tag attributes, they are <em>not</em> added to the closing tag. It's one of those elegant moments where everything just works perfectly. I love this shortcut!</p>
<h2 id="other-keyboard-shortcuts" tabindex="-1">Other Keyboard Shortcuts</h2>
<p>Quickly, here's a list of other built-in shortcuts that I find myself doing all day, every day:</p>
<ul>
<li>Super+K+B toggles the file navigator</li>
<li>Super+P opens file search. File search is <em>super</em> powerful!
<ul>
<li>Fuzzy matching: you don't have to spell perfectly or even do consecutive chunks. Say I wanted to open "RecruitWeb.less" but I have 3 of them in different folders, named main, mobile, and new. If I hit Super+P and start typing "mobrecrless" it matches mobile/RecruitWeb.less.</li>
<li>Line-jumps: Jumping into a file to fix a bug and you know the line number? select your file with fuzzy matching then add a colon and the line number: "mobrecrless:47" puts your cursor on line 47 of the file you choose.</li>
<li>Symbol search: Instead of using a colon to jump to a specific line, say you know the CSS selector you want to modify but you don't know where it is in the file. Use "@" and start typing the selector: "mobrecrless@.ui-widget"</li>
</ul>
</li>
<li>Super+Shift+P opens the command palette, from which you can run package control, or toggle the minimap, or execute just about any command for which you can't remember the keyboard shortcut. This uses fuzzy matching as well.</li>
<li>Super+Alt+Left/Right flip between open tabs</li>
</ul>
<h2 id="plugins-without-which-i-can't-live" tabindex="-1">Plugins Without Which I Can't Live</h2>
<ul>
<li><strong><a href="http://wbond.net/sublime_packages/package_control">Package Control</a>:</strong> Don't leave home without it! Use this to install and enable/disable other plugins. Very handy! And the reason I won't include links to the rest of these plugins, because you can just search for them on Package Control.</li>
<li><strong>Sidebar Enhancements:</strong> Adds the aforementioned F2 to rename, confirm deletes, and many other goodies</li>
<li><strong>Advanced New File:</strong> Super+Alt+N to create a new file and you can type out its path relative to the project root, so if I have a project open and type "assets/less/style.less" it will create style.less at the specified location within the project, creating any of the folders necessary in that path (and open the file for editing).</li>
<li><strong>Gist:</strong> I store all of my snippets in <a href="https://gist.github.com/atuttle-snippets">their own github account</a> so that they are not tied to my editor, and they can be shared, forked, and collaborated on. There is a small latency initiating an "insert gist" command while it indexes my available snippets, but I find that I use them infrequently enough that it doesn't bother me. If I used them more regularly I would probably do something local. Using gists also means I don't have to use the sublime snippet syntax, which is not difficult, but it is another set of (minor) "lock-in" that I would have to go back and remove if/when I move on to another editor in the future.</li>
<li><strong>Live Reload:</strong> because refreshing pages to see your changes is for chumps.</li>
<li><strong>Markdown:</strong> I write a lot of things in <a href="http://daringfireball.net/projects/markdown/">markdown</a>, including blog posts here and elsewhere, articles for things like <a href="http://learncfinaweek.com/">Learn CF In A Week</a>, and so on. It's become a core part of how I write. This plugin adds syntax highlighting for Markdown.</li>
</ul>
<p>I think that about covers it... for now. I seem to be continuously evolving how I use Sublime, which is a tremendous part of what makes it so useful: you can mold it to the way you like to work.</p>
Happy 2nd Birthday Taffy!2012-08-23T08:08:26Zhttps://adamtuttle.codes/blog/2012/Happy-2nd-Birthday-Taffy/<p>It's hard to believe but it was 2 years ago today that I finally <a href="https://adamtuttle.codes/blog/2010/Taffy-A-Restful-Framework-for-ColdFusion/">declared Taffy to be at 1.0 and released it to the public</a>. It's come a long way since then, and I'm working hard on the next version. I'm still not sure if that will be 1.2 or 2.0, but there is plenty on the horizon and Taffy seems to be gaining more traction every week.</p>
<p>Taffy has definitely been my most widely-used open source project, with the possible exception of one or two of my most popular Mango Blog plugins, but certainly it's been the one into which I've put the most effort. I'm thrilled that so many people find it useful, and that I have been able to use it to provide data to all of the <a href="https://play.google.com/store/apps/details?id=com.countermarch.casemobile">mobile</a> <a href="http://itunes.apple.com/us/app/case-mobile/id520119520?mt=8">applications</a> that <a href="https://play.google.com/store/apps/details?id=com.countermarch.MITReunionMobile">I've</a> <a href="http://itunes.apple.com/us/app/mit-tech-reunions-2012/id528581324?mt=8">built</a> in the last 6 months.</p>
<p>Here's to many more years to come!</p>
Using bCrypt in ColdFusion 102012-06-06T00:00:00Zhttps://adamtuttle.codes/blog/2012/using-bcrypt-in-coldfusion-10/<p>What follows is an article <a href="https://web.archive.org/web/20160531090150/http://www.aliaspooryorik.com/blog/index.cfm/e/posts.details/post/using-bcrypt-in-coldfusion-10-370">originally written by John Whish</a>, reproduced here with his permission. I'm re-posting it because his blog has since gone offline and I like linking to this information, including to <a href="https://restassuredbook.com/">in my book, REST Assured</a>. I'm posting this from the future <strong>(2021! The hoverboards are everything we ever dreamed of and more! <img src="https://adamtuttle.codes/img/2012/hoverboard.gif" alt="hover boards omg!" style="max-height: 1.4em; margin: 0 4px -7px 0" />)</strong> but back-dating the entry to match John's original post date.</p>
<hr />
<p>As you may have seen on the twitterverse, it has been reported that <a href="https://news.ycombinator.com/item?id=4073309">6.5 Million LinkedIn Password Hashes Leaked</a>. There are several comments about using bCrypt (or sCrypt) to provide improved security and this reminded me that Marc Esher blogged about <a href="http://blog.mxunit.org/2011/02/hashing-passwords-with-bcrypt-in.html">Hashing passwords with bcrypt in ColdFusion</a>. Since Marc posted this, ColdFusion 10 has been released which has improved Java integration. so I thought I'd take his code and do it using ColdFusion 10 and here it is:</p>
<p><strong>Application.cfc</strong></p>
<pre class="language-js"><code class="language-js">component <span class="token punctuation">{</span><br /> <span class="token keyword">this</span><span class="token punctuation">.</span>name <span class="token operator">=</span> <span class="token string">"bcrypt_in_cf10"</span><span class="token punctuation">;</span><br /> <span class="token keyword">this</span><span class="token punctuation">.</span>javaSettings <span class="token operator">=</span> <span class="token punctuation">{</span><br /> LoadPaths <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token string">"/classfiles"</span><span class="token punctuation">]</span><br /> <span class="token punctuation">}</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>I also needed to create a directory called "classfiles" and drop in the BCrypt.class script (which I downloaded from <a href="https://s3.amazonaws.com/marc.esher/BCrypt.class">here</a>)</p>
<p><strong>bcrypttest.cfm</strong></p>
<pre class="language-js"><code class="language-js"><span class="token operator"><</span>cfscript<span class="token operator">></span><br /> bcrypt <span class="token operator">=</span> <span class="token function">CreateObject</span><span class="token punctuation">(</span> <span class="token string">"java"</span><span class="token punctuation">,</span> <span class="token string">"BCrypt"</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">WriteDump</span><span class="token punctuation">(</span> bcrypt <span class="token punctuation">)</span><span class="token punctuation">;</span> pw <span class="token operator">=</span> <span class="token string">"happy1.!gIlm0re"</span><span class="token punctuation">;</span> startts <span class="token operator">=</span> <span class="token function">getTickCount</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> hashed <span class="token operator">=</span> bcrypt<span class="token punctuation">.</span><span class="token function">hashpw</span><span class="token punctuation">(</span>pw<span class="token punctuation">,</span> bcrypt<span class="token punctuation">.</span><span class="token function">gensalt</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">writeoutput</span><span class="token punctuation">(</span><span class="token string">"created password '"</span> <span class="token operator">&</span> hashed <span class="token operator">&</span> <span class="token string">"' in "</span> <span class="token operator">&</span> <span class="token function">getTickCount</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span><br /> startts <span class="token operator">&</span> <span class="token string">" ms"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> startts <span class="token operator">=</span> <span class="token function">getTickCount</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> match <span class="token operator">=</span> bcrypt<span class="token punctuation">.</span><span class="token function">checkpw</span><span class="token punctuation">(</span>pw<span class="token punctuation">,</span> hashed<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">writeoutput</span><span class="token punctuation">(</span>"checked pw <span class="token function">match</span><br /> <span class="token punctuation">(</span>#match#<span class="token punctuation">)</span> <span class="token keyword">in</span> <span class="token string">" & getTickCount() - startts & "</span> ms"<span class="token punctuation">)</span><span class="token punctuation">;</span> startts <span class="token operator">=</span> <span class="token function">getTickCount</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> hashed <span class="token operator">=</span> bcrypt<span class="token punctuation">.</span><span class="token function">hashpw</span><span class="token punctuation">(</span>pw<span class="token punctuation">,</span><br /> bcrypt<span class="token punctuation">.</span><span class="token function">gensalt</span><span class="token punctuation">(</span><span class="token number">12</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">writeoutput</span><span class="token punctuation">(</span><span class="token string">"created password '"</span> <span class="token operator">&</span> hashed <span class="token operator">&</span> <span class="token string">"' in "</span> <span class="token operator">&</span> <span class="token function">getTickCount</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> startts <span class="token operator">&</span> <span class="token string">" ms"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> startts<br /> <span class="token operator">=</span> <span class="token function">getTickCount</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> match <span class="token operator">=</span> bcrypt<span class="token punctuation">.</span><span class="token function">checkpw</span><span class="token punctuation">(</span>pw<span class="token punctuation">,</span> hashed<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">writeoutput</span><span class="token punctuation">(</span><span class="token string">"checked password match (#match#) in "</span> <span class="token operator">&</span><br /> <span class="token function">getTickCount</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> startts <span class="token operator">&</span> <span class="token string">" ms"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// just for giggles try an incorrect password startts = getTickCount(); match =</span><br /> bcrypt<span class="token punctuation">.</span><span class="token function">checkpw</span><span class="token punctuation">(</span><span class="token string">"5p1na1.Tap"</span><span class="token punctuation">,</span> hashed<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">writeoutput</span><span class="token punctuation">(</span><span class="token string">"checked incorrect password match (#match#) in "</span> <span class="token operator">&</span> <span class="token function">getTickCount</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span><br /> startts <span class="token operator">&</span> <span class="token string">" ms"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token operator"><</span><span class="token operator">/</span>cfscript<span class="token operator">></span></code></pre>
<p>Pretty simple huh!</p>
How You Can Contribute to Taffy Documentation2012-04-13T12:57:50Zhttps://adamtuttle.codes/blog/2012/How-you-can-contribute-to-Taffy-documentation/<p>I've had more than one person offer to help me with Taffy documentation lately, and I would love to be able to open it up just like the source code of the project so that people could make proposed changes and submit a pull request. Unfortunately this is <a href="https://github.com/gollum/gollum/issues/265">a standing request that's yet to be filled for Github Wiki's</a>.</p>
<p>I think I may have found a workaround.</p>
<p>It takes a little bit of extra effort (only a little!) and I would only recommend it for intermediate gitter's and above. (<em>Sorry noobs!</em>)</p>
<p>Every wiki should be available as a git repo already. For example, you can see Taffy's <a href="https://github.com/atuttle/Taffy/wiki/_access">here</a>, and FW/1's <a href="https://github.com/seancorfield/fw1/wiki/_access">here</a>.</p>
<p>My proposed workflow is this:</p>
<ol>
<li>Manually create a fork of the Taffy wiki on your Github account:
<ul>
<li>Create a new repository on your github account. Let's call it "Taffy-Wiki".</li>
<li>Clone the Taffy wiki repo to your local machine somewhere:<br /><code>git clone git://github.com/atuttle/Taffy.wiki.git</code></li>
<li>Remove the original "origin" remote and add your github repo as new "origin"<br /><code>git remote rm origin</code> and<br /><code>git remote add origin git://github.com/<YOUR_USERNAME>/Taffy-Wiki.git</code></li>
</ul>
</li>
<li>Make your proposed changes locally, then push them to your github account:<br /><code>git push -u origin master</code><br />('-u origin master' only required the first time; afterwards just do <code>git push</code>)</li>
<li>Submit a ticket to the <a href="https://github.com/atuttle/Taffy/issues">official Taffy issue tracker</a> requesting me to review your changes and merge them in. <strong>Please be sure to include a link to your repo and describe what you've changed.</strong></li>
<li>Goto #2</li>
</ol>
<p>I apologize if that sounds complicated, but when you think about it in comparison to making code changes and submitting a pull request, it's not <em>that much</em> more complex. Also, here's a script you can copy/paste into your terminal that will do the forking for you, except the repo creation:</p>
<pre><code>git clone git://github.com/atuttle/Taffy.wiki.git
cd Taffy.wiki
git remote rm origin
git remote add origin git://github.com/<YOUR_USERNAME>/Taffy-Wiki.git
</code></pre>
<p>This requires that you:</p>
<ul>
<li>First create a new Github repo named "Taffy-Wiki"</li>
<li>Insert your github username for <code><YOUR_USERNAME></code> toward the end</li>
</ul>
<p>This way anyone can submit documentation patches and I'll be able to review easily merge them in and keep the published wiki updated. (I had considered just putting the wiki source in to a folder or branch of the source repo, or its own repo, but that comes with its own challenges like keeping the published wiki updated.)</p>
<p>Thanks for your interest and contributions!</p>
Taffy 1.1 RC1 Available2011-10-27T12:30:35Zhttps://adamtuttle.codes/blog/2011/Taffy-1-1-RC1-Available/<p>I've been working hard, in my nooks and crannies of free time, to put together and release Taffy 1.1. Lots of stuff has changed for the better, and it's a pretty sweet upgrade, if I do say so myself.</p>
<p>Luckily, I haven't been working alone. To date, I've had community contributions in the form of code, help testing, and advice from (in no particular order other than that with which they came to mind) <a href="http://ghostednotes.com/">Brian Panulla</a>, <a href="http://davejlong.com/blog/">David Long</a>, <a href="http://www.gregmoser.com/blog/">Greg Moser</a>, <a href="http://www.danlance.co.uk/">Dan Lancelot</a>, <a href="http://www.countermarch.com/">Steve Rittler</a>, and <a href="http://www.barneyb.com/barneyblog/">Barney Boisvert</a>. There are more, I'm sure; and I'm deeply sorry for forgetting them/you!</p>
<p>But recently I realized that work has stagnated a little bit. Releasing version 1.1 has been held up for more than 5 months now almost entirely by my desire to add support for Railo. While I'm all for the open source movement (Taffy is open source after all), and the people working on and for Railo are some pretty nice guys, I just can't let that continue. So I'm going to release 1.1 soon, with or without Railo support.</p>
<p>Don't get me wrong, I'd still love to have it supported and I'm happy to review any pull requests anyone sends in. It's just that I've had a hell of a time getting a decent environment setup to run my tests and work out the kinks, and while there has been <em>some</em> support from the Railo community, I'm afraid to say it's just not been enough.</p>
<p>So to that end, I'm posting <strong>Taffy 1.1 Release Candidate 1 (RC1)</strong> here today. You can <a href="https://github.com/atuttle/Taffy/zipball/1.1-rc"><strong>download the zip</strong></a> and play with it, but the best thing you could do would be to clone from <a href="https://github.com/atuttle/Taffy/tree/1.1-rc">the 1.1-rc branch on github</a> to make it easy to get the latest updates. So what's left to do before this release candidate is declared final? Most importantly, I need to finish the documentation updates. I've got a lot done that was published earlier today, and there's still some more to do. And of course, I need people to play with it and <a href="https://github.com/atuttle/Taffy/issues?milestone=1&sort=created&direction=desc&state=open">report any bugs you find!</a></p>
<p>And, if you're in a giving mood, I'll be keeping <a href="https://github.com/atuttle/Taffy/tree/Railo">the Railo branch</a> up to date. This branch is designated for testing and sharing Railo-specific fixes, and then once they are confirmed working in Railo and confirmed not to break against Adobe CF, I'll merge them into whatever the current dev/RC branch is.</p>
<p>I really hope that the Railo community can help me finish what's left to get their platform supported, but in the end it boils down to shipping 1.1 being more important --for me, at least-- than including Railo support.</p>
<p>Look for a post soon outlining some of what's changed since 1.0. I'm pretty excited about it! You can get a preview of that post <a href="https://github.com/atuttle/Taffy/wiki/Roadmap">here on the roadmap</a>, if you just can't wait.</p>
<dl class="plugin-data">
<dt>Taffy version:</dt>
<dd>1.1 rc1</dd>
<dt>Project:</dt>
<dd><a href="https://github.com/atuttle/Taffy">https://github.com/atuttle/Taffy</a></dd>
<dt class="install">Download:</dt>
<dd class="install"><a href="https://github.com/atuttle/Taffy/zipball/1.1-rc">Zip from Github</a></dd>
</dl>
Authenticating Your Taffy-Powered API2011-10-01T14:35:54Zhttps://adamtuttle.codes/blog/2011/Authenticating-your-Taffy-powered-API/<p>Authentication is an important problem, and usually one of the first questions I get when I'm telling people about Taffy. My canned response is that Taffy does not handle any authentication for you, but it does expose some functionality for you to build authentication into your APIs.</p>
<p>I cover the topic a little bit in <a href="https://adamtuttle.codes/blog/2011/My-cfObjective-2011-Slides-Notes/">the presentations I've given on Taffy</a>, but essentially the message is the same.</p>
<p>The reason that Taffy doesn't handle authentication is because there are probably a dozen or more ways you could accomplish it (OpenID, OAuth1, OAuth2, HTTP Basic, and the list goes on...), and no one way would satisfy everyone. Trying to add multiple methods into the framework would add a lot of bulk that would be wasted by anyone not using that functionality.</p>
<p>To help you add authentication, Taffy calls a method in Application.cfc named <strong>onTaffyRequest</strong>, passing in all of the details of the request (verb, headers, arguments, etc) after it has parsed them from the request, but before running any of <em>your</em> code. This allows you to inspect the request and decide whether or not you want to allow it to continue, or redirect, or return an error or... whatever else you might want to do.</p>
<p>That said, people have done a few different things, and I thought it would be useful to share them with you.</p>
<p>Greg Mosier, who is writing an eCommerce plugin for Mura called Slatwall, posted about <a href="http://www.gregmoser.com/blog/ajax-authentication-with-taffy-rest-api/">his solution for authentication</a>, which denies requests that didn't originate from a user with an existing authenticated ColdFusion session.</p>
<p>Then, Glynn Jackson <a href="http://www.cfcoffee.co.uk/index.cfm/2011/9/22/API-Authentication-with-Taffy">posted about his solution</a> which uses public and private keys (kind of like SSH, but not exactly the same) to prevent unauthorized database changes. As discussed in the comments on his post, since he's not using SSL, the data is not secured in-transit; his authentication is only serving to prevent unauthorized reads and writes against his database. Anyone sniffing traffic in the middle would be able to see what the data was. Still a perfectly valid approach though, depending on your requirements.</p>
<p>Of course there are a lot more ways to accomplish authentication, but hopefully these examples will get you started and help you understand how <strong>onTaffyRequest</strong> works.</p>
My cfObjective 2011 Slides & Notes2011-05-31T08:00:53Zhttps://adamtuttle.codes/blog/2011/My-cfObjective-2011-Slides-Notes/<p>It seems like every time I make a promise to get something done for the community -- Taffy 1.1, bug fixes for a Mango plugin, and now this -- life has a way of making me break that promise. To wit, I was (still am) slammed at work with more than the typical workload. I'm not complaining. Job security is nice, but I feel bad for broken promises.</p>
<p>So here -- better late than never right? -- is my slide deck and notes from my cfObjective 2011 session, <a href="http://www.cfobjective.com/index.cfm/sessions/no-nonsense-restful-coldfusion-web-services-with-taffy/">No nonsense RESTful ColdFusion Web Services, with Taffy</a>.</p>
<p>For those of you who attended my Taffy presentations at <a href="http://www.phillycfug.org/">Philly CFUG</a> or on <a href="http://www.coldfusionmeetup.com/">The Online ColdFusion Meetup</a>, it is mostly the same, but it did evolve a little bit. If you want a refresher, this is the most relevant version to watch.</p>
<p>(Sorry, the website that I used to host these slides has since shuttered and the domain is not relevant any more. So long, SlideSix!)</p>
<p>For the slide notes, you should <a href="https://adamtuttle.codes/Taffy-1.1-final.pdf">download this PDF</a>, which has the slide at the top of the page, and the notes at the bottom.</p>
<p>Where is the code? You get it when you download <a href="http://taffy.io/">Taffy</a>. All of the code that I showed, except for <a href="https://gist.github.com/887610">the "hard way" code I briefly show, which is available here</a> (link also in the slides), was examples that come in the examples folder of the Taffy download.</p>
<p>Lastly, <a href="http://cfmumbojumbo.com/cf/">Tim Cunningham</a> was in attendance and kind enough to record my session on video for posterity. I've embedded the two parts below. Thanks, Tim! (Also, thanks for letting me crash on a cot in your room when the hotel gave away my room on Wednesday night!)</p>
<embed src="http://cfmumbojumbo.com/plugins/MuraMediaPlayer_1/cfmediaplayer/player/player.swf" width="480" height="290" allowscriptaccess="always" allowfullscreen="true" flashvars="file=http://c210253.r53.cf1.rackcdn.com/taffy.FLV&stretching=fill" />
<embed src="http://cfmumbojumbo.com/plugins/MuraMediaPlayer_1/cfmediaplayer/player/player.swf" width="480" height="290" allowscriptaccess="always" allowfullscreen="true" flashvars="file=http://c210253.r53.cf1.rackcdn.com/taffy2.FLV&stretching=fill" />
<p>Thanks again for coming to my session. It was my first time presenting at a conference, and to get such a positive response was amazing. I hope you got something valuable from it, and I hope to see you at a future event. :)</p>
Taffy at cfObjective2011-05-11T14:18:26Zhttps://adamtuttle.codes/blog/2011/Taffy-at-cfObjective/<p>I'll be leaving in just a few short hours to fly to Minneapolis and attend <a href="http://www.cfobjective.com/">cfObjective 2011</a>.</p>
<p>In case you didn't already know, I'll be presenting on creating REST web services with ColdFusion, and of course, <a href="http://taffy.riaforge.org/">Taffy</a>. <a href="http://www.cfobjective.com/index.cfm/sessions/no-nonsense-restful-coldfusion-web-services-with-taffy/">My session</a> is <strong>Friday morning at 10:15, in ballroom D3</strong>.</p>
<p>If my laptop battery holds out during my flights, I should be able to finish the last few tasks in time to release Taffy 1.1 at my presentation on Friday morning. Cross your fingers. (If not, it should be within a week or two!)</p>
<p>While I'm babbling about Taffy, I'll also mention that I've created a Google Group for it. It's kind of hard for me to believe, but adoption has been steadily increasing and I am no longer able to respond to every Taffy-related email I get in a timely manner. I'm hoping that users can help each other out via the group -- and of course we'll discuss best practices and the future of the framework there, too. If you're interested, you can find it here: <a href="https://groups.google.com/forum/#!forum/taffy-users">https://groups.google.com/forum/#!forum/taffy-users</a></p>
Enforcing Limitations on Your Taffy-Powered API2010-10-18T08:00:37Zhttps://adamtuttle.codes/blog/2010/Enforcing-Limitations-on-your-Taffy-powered-API/<p>You might be one of those people want to open up access to create, update, and delete your data via your API to the entire world with no restrictions; but it's not likely. A common need for APIs is defining and enforcing limitations such as API keys and authentication, rate limiting, logging, and so on.</p>
<p>Taffy provides a hook that allows you to allow or deny a request to continue, on a per-request basis, with <a href="http://github.com/atuttle/Taffy/wiki/Index-of-API-Methods#onTaffyRequest">onTaffyRequest</a>.</p>
<p>When you define <code>onTaffyRequest</code> in your Application.cfc, it is called for each request and sent all of the information it needs to process the request:</p>
<ul>
<li>HTTP Verb</li>
<li>Name of the CFC that would respond to the request</li>
<li>All request arguments, including tokens extracted from the URI and Query String/Put/Post parameters.</li>
<li>Requested Mime Type (json, xml, etc)</li>
<li>And (as of Taffy 1.1) a structure of the HTTP request headers</li>
</ul>
<p>The default implementation, (when you <em>don't</em> override it) always returns <code>TRUE</code>; and, in the event that you would like to allow the request to continue so that your resources are allowed to do their thing, your implementation of <code>onTaffyRequest</code> should return <code>TRUE</code> as well.</p>
<p>If, on the other hand, you want to abort the request -- because login failed, or the customer owes you money, or they are over their rate limit (these are business rules and up to you to implement) -- then you should return a <strong>representation instance</strong>. This is just an instance of the object that serializes result data into the requested format, except in this case it will be used to tell the API consumer that, and hopefully <em>why</em>, their request is not allowed to continue.</p>
<p>Let's say that you're using the default representation class that comes with Taffy (capable of JSON serialization using ColdFusion's native <code>SerializeJSON</code> method), and you want to abort the current request:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">function</span> <span class="token function">onTaffyRequest</span><span class="token punctuation">(</span><span class="token parameter">verb<span class="token punctuation">,</span> cfc<span class="token punctuation">,</span> requestArguments<span class="token punctuation">,</span> mimeExt<span class="token punctuation">,</span> headers</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token keyword">var</span> o <span class="token operator">=</span> <span class="token function">StructNew</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token comment">/* customer owes you money */</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> o<span class="token punctuation">.</span>msg <span class="token operator">=</span> <span class="token string">'Your account is overdue. Please call Accounts Payable at 555-867-5309.'</span><span class="token punctuation">;</span><br /> <span class="token keyword">return</span> <span class="token function">newRepresentation</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">setData</span><span class="token punctuation">(</span>o<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">withStatus</span><span class="token punctuation">(</span><span class="token number">403</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /> <span class="token comment">//all checks passed, let the request continue</span><br /> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>The method <code>newRepresentation</code> will be available in Taffy 1.1; in the meantime, you will need to instantiate the representation object manually. If your default representation class is the Taffy default (<code>taffy.core.genericRepresentation</code>), then you would do so this way:</p>
<pre class="language-js"><code class="language-js"><span class="token keyword">var</span> r <span class="token operator">=</span> <span class="token function">createObject</span><span class="token punctuation">(</span><span class="token string">'component'</span><span class="token punctuation">,</span> <span class="token string">'taffy.core.genericRepresentation'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token keyword">return</span> r<span class="token punctuation">.</span><span class="token function">setData</span><span class="token punctuation">(</span>o<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">withStatus</span><span class="token punctuation">(</span><span class="token number">403</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
<p>Obviously, substitute your custom representation class cfc path there, if you're using a custom class; so that the result can be serialized to XML, or YAML, or whatever format it is that your API supports and the consumer is expecting their results to be in.</p>
<p>The status code I've used here is 403, which is the official HTTP Status Code for "Not Allowed". You should probably familiarize yourself with <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html">HTTP Status Codes</a>, and maybe print it out and keep it handy. They are an important part in the way REST APIs work.</p>
<p>Doing all of this allows Taffy to respond to the request in a RESTful manner that the consumer software can understand (for example, as json/xml/etc) while still preventing unauthorized access.</p>
Supporting Multiple Data Formats in a Taffy-Powered API2010-10-14T08:00:21Zhttps://adamtuttle.codes/blog/2010/Supporting-Multiple-Data-Formats-in-a-Taffy-powered-API/<blockquote>
<p><strong>Note:</strong> The code used in this example post will be based on the upcoming release of Taffy 1.1. The differences are minor, and the public builds available on Github should always be stable (unless a bug sneaks by me), so I would encourage you to upgrade. I will point out the differences as I go, for those that don't want to use interim builds. Even though 1.1 is not officially released yet, you can make use of changes like those seen here by downloading <a href="http://github.com/atuttle/Taffy">the latest build from Github</a>. (And you need not know anything about Git to do so, it's almost the same as downloading a project from RIAForge...)</p>
</blockquote>
<p>One of the first questions people seem to have about Taffy is about supporting multiple return formats. While this is still fairly simple to do -- once you know how, I guess -- it is probably one of the more complex things to do with Taffy. This post shows example code for supporting both XML and JSON data formats ("mime types") from a single API.</p>
<p>The first thing that you need to understand is that the <strong>Representation Class</strong> is a single component that can handle <em>all</em> of the data formats that your API supports. So if your API needs to support JSON and XML, as this example will, then the component needs to be able to handle both formats.</p>
<p>Here is our example custom representation component, which I've named <strong>CustomRepresentationClass.cfc</strong>, and put into my resources folder:</p>
<pre class="language-xml"><code class="language-xml"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfcomponent</span> <span class="token attr-name">extends</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>taffy.core.baseRepresentation<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfset</span> <span class="token attr-name">variables.JSONUtil</span> <span class="token attr-value"><span class="token punctuation attr-equals">=</span> application.JSONUtil</span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfset</span> <span class="token attr-name">variables.AnythingToXML</span> <span class="token attr-value"><span class="token punctuation attr-equals">=</span> application.AnythingToXML</span> <span class="token punctuation">/></span></span><br /><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cffunction</span><br /> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>getAsJSON<span class="token punctuation">"</span></span><br /> <span class="token attr-name"><span class="token namespace">taffy:</span>mime</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>application/json<span class="token punctuation">"</span></span><br /> <span class="token attr-name"><span class="token namespace">taffy:</span>default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>true<span class="token punctuation">"</span></span><br /> <span class="token attr-name">output</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfreturn</span> <span class="token attr-name">variables.JSONUtil.serializeJson(variables.data)</span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cffunction</span><span class="token punctuation">></span></span><br /><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cffunction</span><br /> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>getAsXML<span class="token punctuation">"</span></span><br /> <span class="token attr-name"><span class="token namespace">taffy:</span>mime</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>application/xml<span class="token punctuation">"</span></span><br /> <span class="token attr-name">output</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfreturn</span> <span class="token attr-name">variables.AnythingToXML.ToXML(variables.data)</span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cffunction</span><span class="token punctuation">></span></span><br /><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cfcomponent</span><span class="token punctuation">></span></span></code></pre>
<p>There are a few things of note that have changed for 1.1:</p>
<ul>
<li>
<p>The component extends <code>taffy.core.baseRepresentation</code>, not <code>taffy.core.genericRepresentation</code>. The <code>genericRepresentation</code> class is still available for backward compatibility, but future code should use <code>baseRepresentation</code>.</p>
</li>
<li>
<p>The <code>taffy_mime</code> and <code>taffy_default</code> metadata attributes of the function tags have been replaced by <code>taffy:mime</code> and <code>taffy:default</code> respectively. The old names are still supported, though deprecated.</p>
</li>
</ul>
<p>And here are the other things that are important about the above code:</p>
<ul>
<li>
<p>Both "getAsJSON" and "getAsXML" functions are implemented; which tells Taffy that both of these formats are supported by your API, and it will allow ".json" and ".xml" extensions to be used in the URI.</p>
</li>
<li>
<p>The libraries used, <a href="http://jsonutil.riaforge.org/">JSONUtil</a> and <a href="http://anythingtoxml.riaforge.org/">AnythingToXML</a>, are not instantiated by this component; they are stored (cached) in the Application scope and referenced by this component. This is for optimal performance.</p>
</li>
</ul>
<p>There are two other parts to this puzzle: Putting the libraries in the right location, and caching them in the Application scope for your custom representation class to use.</p>
<p>There is no "right" place to put the libraries, but I like to put them into the resources folder. For the sake of cleanliness, I like to put an AnythingToXML folder inside the resources folder and put the AnythingToXML components inside that folder, because there are a few of them. In the case of JSONUtil, which is only a single component, you could just put the component directly into the resources folder; but for consistency, I create a JSONUtil folder and put the JSONUtil component inside it. Now, our example API folder structure resembles this:</p>
<p><img src="https://adamtuttle.codes/img/2010/taffy-multimime-libs.png" alt="Folder structure for supporting multiple mime types" /></p>
<p>All that's left to do is cache the libraries in Application scope. My example Application.cfc is:</p>
<pre class="language-xml"><code class="language-xml"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfcomponent</span> <span class="token attr-name">extends</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>taffy.core.api<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfscript</span><span class="token punctuation">></span></span><br /> this.name = hash(getCurrentTemplatePath());<br /><br /> function applicationStartEvent(){<br /> application.JsonUtil = createObject("component", "resources.JSONUtil.JSONUtil");<br /> application.AnythingToXML = createObject("component", "resources.AnythingToXML.AnythingToXML");<br /> }<br /><br /> function configureTaffy(){<br /> setDefaultRepresentationClass("resources.CustomRepresentationClass");<br /> }<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cfscript</span><span class="token punctuation">></span></span><br /><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cfcomponent</span><span class="token punctuation">></span></span></code></pre>
<p>In the <code>applicationStartEvent</code> method, we create and cache the library objects in the application scope, and in the <code>configureTaffy</code> method, we set the default representation class to be our custom class instead of the framework default.</p>
<p>That's it. Your API will now respond to requests for foo.json with a JSON representation of foo, and foo.xml with an XML representation of foo.</p>
<p>I hope to officially release Taffy 1.1 by early November.</p>
Taffy: A Restful Framework for ColdFusion2010-08-23T11:00:13Zhttps://adamtuttle.codes/blog/2010/Taffy-A-Restful-Framework-for-ColdFusion/<blockquote>
<p><strong>Update:</strong> Since this was posted in August of 2010, a lot has changed with Taffy. We're now approaching a 1.2 release, and new development should not be done against 1.0. <a href="http://atuttle.github.com/Taffy/">See this page for more information</a>.</p>
</blockquote>
<p>Today I'm excited to announce something that I've been working on quietly for a few months. I've created a framework for the purpose of building Restful APIs with semantically correct URIs -- no easy task in vanilla CFML -- simply and elegantly. I call it <a href="http://taffy.riaforge.org/">Taffy</a>.</p>
<p>I've taken some inspiration from <a href="http://fw1.riaforge.org/">FW/1</a>, <a href="http://powernap.riaforge.org/">Powernap</a>, and <a href="http://swizframework.org/">Swiz</a>. Much like FW/1, Taffy is a set of base classes that your Application.cfc and resource CFCs extend; and similar to Powernap, Taffy uses the notion of "representation" classes. Don't worry about these yet - it's easy and I'll get into them in more detail later. Lastly, using Swiz taught me the power of metadata, so I made it a big part of Taffy.</p>
<p>As much as is possible, Taffy uses convention instead of configuration. There's no messy mapping of <em><strong>this</strong> http verb used with <strong>this</strong> URI calls <strong>this</strong> function in <strong>this</strong> cfc</em>. You set a few bits of metadata and use a few pre-determined function names, and everything falls into place.</p>
<p>Enough blabber, let's look at a REST API written with Taffy!</p>
<h2 id="setting-up-the-framework" tabindex="-1">Setting up the framework</h2>
<p><strong>Application.cfc</strong></p>
<pre class="language-js"><code class="language-js">component <span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.api"</span> <span class="token punctuation">{</span><br /> <span class="token keyword">this</span><span class="token punctuation">.</span>name <span class="token operator">=</span> <span class="token function">hash</span><span class="token punctuation">(</span><span class="token function">getCurrentTemplatePath</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><span class="token punctuation">}</span></code></pre>
<p>Simple enough for you? In case you missed it, this is a generic application with a unique name... and it uses the component <strong>extends</strong> attribute to extend Application.cfc from Taffy's core api class.</p>
<p>The only requirement is that you either put Taffy in your web root, or create a CF mapping to it.</p>
<p>Ok, so there are <em>a few</em> configuration options, all of which you set in Application.cfc, as in the next code sample. I'm going to show all of the configuration options, but use their default values.</p>
<pre class="language-js"><code class="language-js">component <span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.api"</span> <span class="token punctuation">{</span><br /><br /> <span class="token keyword">this</span><span class="token punctuation">.</span>name <span class="token operator">=</span> <span class="token function">hash</span><span class="token punctuation">(</span><span class="token function">getCurrentTemplatePath</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /><br /> <span class="token comment">//use this instead of onApplicationStart()</span><br /> <span class="token keyword">void</span> <span class="token keyword">function</span> <span class="token function">applicationStartEvent</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><span class="token punctuation">}</span><br /><br /> <span class="token keyword">void</span> <span class="token keyword">function</span> <span class="token function">configureTaffy</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span><br /> <span class="token comment">//these are all default values, but if you want to change them this is the place:</span><br /> <span class="token function">setDebugKey</span><span class="token punctuation">(</span><span class="token string">"debug"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">setReloadKey</span><span class="token punctuation">(</span><span class="token string">"reload"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">setReloadPassword</span><span class="token punctuation">(</span><span class="token string">"true"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">setDashboardKey</span><span class="token punctuation">(</span><span class="token string">"dashboard"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">setDefaultRepresentationClass</span><span class="token punctuation">(</span><span class="token string">"taffy.core.genericRepresentation"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">registerMimeType</span><span class="token punctuation">(</span><span class="token string">"json"</span><span class="token punctuation">,</span> <span class="token string">"application/json"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token function">setDefaultMime</span><span class="token punctuation">(</span><span class="token string">"json"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><br /> <span class="token punctuation">}</span><br /><span class="token punctuation">}</span></code></pre>
<p>Still pretty simple, right? And now you have an application capable of servicing REST requests. You just don't have any resources.</p>
<h2 id="resources" tabindex="-1">Resources</h2>
<p>In general with REST, there are two types of resources: Collections and Members. Think of Collections as a query object that shows the entire contents of the table, and Members as a structure representing a single row of the same query. With Taffy, you implement Collections and Members as separate CFCs.</p>
<p>Let's start with a Collection, using the trusty old Art Gallery datasource:</p>
<p><strong>artistCollection.cfc</strong></p>
<pre class="language-xml"><code class="language-xml"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfcomponent</span> <span class="token attr-name">extends</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>taffy.core.resource<span class="token punctuation">"</span></span> <span class="token attr-name">taffy_uri</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>/artists<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cffunction</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>get<span class="token punctuation">"</span></span> <span class="token attr-name">access</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>public<span class="token punctuation">"</span></span> <span class="token attr-name">output</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfset</span> <span class="token attr-name">var</span> <span class="token attr-name">q</span> <span class="token attr-value"><span class="token punctuation attr-equals">=</span> <span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfquery</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>q<span class="token punctuation">"</span></span> <span class="token attr-name">datasource</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cfartgallery<span class="token punctuation">"</span></span> <span class="token attr-name">cachedwithin</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#createTimeSpan(0,0,0,1)#<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> select * from artists<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cfquery</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfreturn</span> <span class="token attr-name">representationOf(q).withStatus(200)</span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cffunction</span><span class="token punctuation">></span></span><br /><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cffunction</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>post<span class="token punctuation">"</span></span> <span class="token attr-name">access</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>public<span class="token punctuation">"</span></span> <span class="token attr-name">output</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>firstname<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>lastname<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>address<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>city<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>state<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>postalcode<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>email<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>phone<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>fax<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfargument</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>thepassword<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>string<span class="token punctuation">"</span></span> <span class="token attr-name">required</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>false<span class="token punctuation">"</span></span> <span class="token attr-name">default</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfset</span> <span class="token attr-name">var</span> <span class="token attr-name">q</span> <span class="token attr-value"><span class="token punctuation attr-equals">=</span> <span class="token punctuation">"</span><span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfquery</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>q<span class="token punctuation">"</span></span> <span class="token attr-name">datasource</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cfartgallery<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> insert into artists (firstname,lastname,address,city,state,postalcode,email,phone,fax,thepassword)<br /> values (<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.firstname#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span>,<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.lastname#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span>,<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.address#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span>,<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.city#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span>,<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.state#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span>,<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.postalcode#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span>,<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.email#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span>,<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.phone#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span>,<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.fax#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span>,<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.thepassword#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> )<br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cfquery</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfquery</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>q<span class="token punctuation">"</span></span> <span class="token attr-name">datasource</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cfartgallery<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> select * from artists<br /> where<br /> firstname = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.firstname#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> and lastname = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.lastname#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> and address = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.address#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> and city = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.city#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> and state = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.state#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> and postalcode = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.postalcode#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> and email = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.email#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> and phone = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.phone#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> and fax = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.fax#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> and thepassword = <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfqueryparam</span> <span class="token attr-name">cfsqltype</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>cf_sql_varchar<span class="token punctuation">"</span></span> <span class="token attr-name">value</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>#arguments.thepassword#<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cfquery</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfreturn</span> <span class="token attr-name">representationOf(q).withStatus(200)</span> <span class="token punctuation">/></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cffunction</span><span class="token punctuation">></span></span><br /><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cfcomponent</span><span class="token punctuation">></span></span></code></pre>
<p>There are quite a few lines of code there, but most of them are querying the database. If we ignore those lines, we go down from about 55 lines to about 21. Let's break down the important stuff:</p>
<ol>
<li>The component extends <strong>taffy.core.resource</strong>. This is where it inherits functionality like the <strong>representationOf</strong> and <strong>withStatus</strong> functions.<br /><br /></li>
<li>The component also has a metadata attribute named <strong>taffy_uri</strong> that tells Taffy the URI of the <em>collection</em> that we are defining. We'll talk more about URIs and how you can use them in the next section.<br /><br /></li>
<li>Functions defined here are named GET and POST. For the sake of discussion, I refer to these as Responders. Notice that these correspond to HTTP verb names (get, put, post, delete). I bet you can guess the convention. That's right, when the GET verb is used, the GET function will be called. <br /><br /></li>
<li>All responders must return using the function <strong>representationOf</strong>. This returns a special class containing your data that can be serialized into multiple formats. A JSON serializing representation class comes with the framework, but you can also create and use a custom representation class capable of serializing to any format you want (for example: XML, YAML, WDDX, HTML, etc). I will cover using a custom representation class in a later post, and may decide to create examples that serialize to various formats.<br /><br /></li>
<li>Notice that responders are not present for all 4 HTTP verbs. If a request is made and no responder exists corresponding to the request verb, Taffy will respond with HTTP status <strong>405 Not Allowed</strong>.</li>
</ol>
<h2 id="uris-and-making-them-work-for-you" tabindex="-1">URIs and making them work for you</h2>
<p>You know what a URI is, right? It's a neat (as in tidy) little chunk of URL that tells the web server <em>exactly</em> what bit of data you're interested in. For example, here's a URI for a recent tweet of mine on twitter:</p>
<pre><code>/AdamTuttle/status/21767382574
</code></pre>
<p>When sent to <a href="http://twitter.com/">Twitter.com</a>, they reply with <a href="http://twitter.com/AdamTuttle/status/21767382574">just that one tweet</a>. In actuality, your browser is making a GET request to Twitter's API, which is responding with some HTML. Likewise, using the URI <code>/AdamTuttle</code> you'll get an HTML response of the entire <em>collection</em> of my tweets (paged).</p>
<p>Taffy makes doing this with ColdFusion simple and powerful. This is where that component metadata comes into play. By setting the <strong>taffy_uri</strong> of <code>/artists</code>, Taffy knows to connect requests for the <code>/artists</code> collection up to our artistCollection CFC.</p>
<p>That's great for collections, but if you're thinking ahead then you've no doubt wondered how we're going to handle unique bits in the URI. Given the same Twitter-based example, we can't handle every permutation of unique tweet ID's in the metadata individually. For one, it's not possible, and even if it were, it's just not a reasonable approach. That's where tokens come in.</p>
<p>This is a perfectly valid <strong>taffy_uri</strong>: <code>/artists/{artistId}</code> -- and in fact, that is the <strong>taffy_uri</strong> of the artistMember CFC.</p>
<p>If we were writing the Twitter API and the tweet member CFC, it might start out like this:</p>
<pre class="language-js"><code class="language-js">component<br /> <span class="token keyword">extends</span><span class="token operator">=</span><span class="token string">"taffy.core.resource"</span><br /> taffy_uri<span class="token operator">=</span><span class="token string">"/{username}/status/{tweetId}"</span> <span class="token punctuation">{</span></code></pre>
<p>When Taffy starts up, it inspects all of your Resource CFCs, grabbing that metadata, and converting the <strong>taffy_uri</strong> to a regular expression used to match incoming requests. All that you need to know is that if your URI defines a token <code>{foo}</code> then it will be passed by name to your responders -- so you have to have an argument named the same as your token, in this case, "foo".</p>
<p>There is one caveat with tokenized URIs. Stripped of all tokens, they must all be unique. If you define two Resources with the URIs <code>/foo/{bar}</code> and <code>/foo/{fubar}</code> then there is no way for Taffy to tell which is the appropriate CFC to use for the request. To Taffy, they both look like this: <code>/foo/{some_token}</code>, so having two of them is not unique, and Taffy will throw an exception during startup.</p>
<h2 id="providing-resources-to-taffy" tabindex="-1">Providing Resources to Taffy</h2>
<p>There are three methods for getting Resources into Taffy.</p>
<ul>
<li>Taffy will automatically load your Resource CFCs from a child folder of the API root named "resources". If you are using ColdFusion 6, 7, or 8, you will need another CF mapping named "/resources" which points to your resources folder, and in CF8, of course, you can use an application-specific mapping. In CF9 (and presumably later) no "/resources" mapping is necessary. This is due to a difference in the way that CFC paths are evaluated when used from a parent CFC, which I plan to write about in a later post.</li>
</ul>
<p>Taffy will also work with an external bean factory, like ColdSpring, in two ways.</p>
<ul>
<li>
<p>First, you can use it to manage your resources <em>instead of</em> putting them into a child folder named "resources". If you choose to do this, there is no need to notify Taffy about each resource in any way; you simply set the bean factory and Taffy will request any beans that extend the taffy.core.resource base class. If you choose to go this route, you can simply manage any dependencies of your resources -- e.g. the service objects they might use to access the database -- in your bean factory configuration.</p>
</li>
<li>
<p>Alternately, you can use the convention folder ("resources") <em>and</em> an external bean factory, which will be used to resolve dependencies. This works similarly to ColdSpring -- if your Resource CFC contains a method named "setFoo" and your bean factory has a bean named "foo", then the "foo" bean will be set into the Resource when it is instantiated and cached.</p>
</li>
</ul>
<p>In both cases, you can set your Taffy API up to be a child-application of a larger application that already uses a bean factory. By doing so, you don't have to duplicate your bean factory configuration, and Taffy will use the same existing factory instance.</p>
<h2 id="download" tabindex="-1">Download</h2>
<p>I suppose that's enough information to get you started. The download comes with several examples: an API implemented using no external bean factory, one using <em>only</em> ColdSpring, and one using <em>both</em> the internal resource factory and ColdSpring. It also has a consumer application that you can point at any of the three example APIs.</p>
<p>There are a few other things that I decided to leave out here for the sake of brevity (ha!), but everything should be pretty well documented in the wiki: <a href="http://github.com/atuttle/Taffy/wiki">http://github.com/atuttle/Taffy/wiki</a>. If you have any questions, just ask!</p>
<p>The project is listed on RIAForge: <a href="http://taffy.riaforge.org/">http://taffy.riaforge.org</a>, and the source is hosted on Github: <a href="http://github.com/atuttle/Taffy">http://github.com/atuttle/Taffy</a>.</p>
<p>You can download as:</p>
<p><a href="https://github.com/atuttle/Taffy/zipball/master"><img alt="Download as ZIP" src="https://github.com/images/modules/download/zip.png" /></a>
<a href="https://github.com/atuttle/Taffy/tarball/master"><img alt="Download as TAR" src="https://github.com/images/modules/download/tar.png" /></a></p>
Optimizing Your Application.cfm2007-04-27T00:00:00Zhttps://adamtuttle.codes/blog/2007/optimizing-your-applicationcfm/<p>Because Application.cfm is called on every request, it is extremely important that it be succinct, efficient, and optimized. Besides containing hundreds of lines of code, the worst and most common faux-pas I've seen among my clients is setting all of your application variables on every request.</p>
<p>Global availability within your application and the fact that they are stored in memory long term are the two greatest advantages to using application variables, and if you are resetting their values on every request you've eliminated half of those. You should instead check for the existence of your application variables and, if not found, reset them.</p>
<p>In short, this is bad code:</p>
<pre class="language-xml"><code class="language-xml"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfapplication</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>myApp<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cflock</span> <span class="token attr-name">scope</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>application<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>exclusive<span class="token punctuation">"</span></span> <span class="token attr-name">timeout</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>30<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfset</span> <span class="token attr-name">application.myvar</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>25<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cflock</span><span class="token punctuation">></span></span></code></pre>
<p>And this is good code:</p>
<pre class="language-xml"><code class="language-xml"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfapplication</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>myApp<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfif</span> <span class="token attr-name">not</span> <span class="token attr-name">isDefined("application.varsAreSet")</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cflock</span> <span class="token attr-name">scope</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>application<span class="token punctuation">"</span></span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>exclusive<span class="token punctuation">"</span></span> <span class="token attr-name">timeout</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>30<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfset</span> <span class="token attr-name">application.myVar</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span>25</span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"><</span>cfset</span> <span class="token attr-name">application.varsAreSet</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>true<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><br /> <span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cflock</span><span class="token punctuation">></span></span><br /><span class="token tag"><span class="token tag"><span class="token punctuation"></</span>cfif</span><span class="token punctuation">></span></span></code></pre>
<p>I am planning on posting additional Application.cfm optimization techniques next week - focusing on how best to make a single application.cfm that works in all of your environments, and as always, welcome questions.</p>