tl;dr: you can embed a robots.txt file into an audio file, and it can be both a valid robots.txt and a valid audio file.
The robots.txt standard has a nifty feature: it will ignore anything that it doesn’t understand. Random text, ASCII art, and even other data can be added without making the robots.txt file invalid.
You could do something simple.
Roses are red
Violets are blue
user-agent: Googlebot
disallow: /foo
Or do something weird.
Sometimes you might find, that after a CMS migration, some things are missing. Obviously nobody would do such a thing on purpose, right? But it can happen. Suddenly, instead of something complete, you’re left with just guitars.
Let’s do a quick & dirty hack to check for important, lost URLs.
curl "http://web.archive.org/cdx/search/cdx?url=colored.house.com*&output=txt&from=20241201&to=20241231" --output - \
| grep " text/html 200 " | awk '{print $3}' | sed 's/\?.*//' | sort | uniq -c | sort -nr \
| awk '{print $2}' | head -n 50 | xargs -I {} curl -s -o /dev/null -w "%{http_code} %{url_effective}\\n" {} \
| grep -v "^[23]"
There you are, the top “important” URLs from the site that worked in December, but which don’t work now (neither redirecting nor returning content). A serious site owner could then check the URLs for things that they either want to maintain (perhaps historical content), if there are new URLs to replace (perhaps one department is now replaced with another), or maybe they just want to keep things 404. SEOs might want to maintain the value of some of the old URLs, even if the replacements are just kinda close.
WLED 0.15.x needs a bit more app space than some ancient WLED installation.
Usually updating the partition table requires a local USB connection.
This is an app that you can install via OTA which expands app partitions to 1536KB.
The general approach to creating partitions on an ESP32 device involve connecting it via USB and flashing it with the IDF library (using gen_esp32part.py to create a binary image of the partition table, and idf.py to flash the device - see the ESP docs. …Or using platformio or similar to do it all together). Changing partition tables without having a device locally connected is not really documented. But you can do it.
Updating partition tables via OTA on ESP32 for WLED 0.15.x »
Midjourney is fun, but the Discord-based interface is somewhat annoying, if you want to make a lot of images. To get all the images you generate, you have to upscale all 2x2 previews, and save them manually. It can get pretty tiring, so instead of making better images, I spent a bit of time to automate the download and import process. With this code, a Discord bot will monitor your server, watch for Midjourney messages, process all images, and upload them to your Lightroom CC cloud account. Run the script anywhere, a Raspberry Pi or a cheap cloud server, and keep making images.
Midjourney / Discord to Lightroom automation - step by step - for 2023 »
While it’s cool to host your own Mastodon server, it’s not really efficient. A simple thing you can do, however, is to make it easier for others to discover your presence, if they already know your website. Try it out – search for “johnmu@johnmu.com” in your Mastodon instance. Also, click “follow” for SEO tips & other bad takes.
Why is hosting your own instance inefficient? In short, sending updates between users (followers, followees) means connections between their individual servers, and if everyone has their own server, it’s a lot of network activity. It costs you money, it costs them money. It’s better if you share instances (and support your server).
Mastodon - be findable with your domain (on Firebase hosting) »
Now that Mastodon is more of a thing, I thought I’d set up a static embed of Mastadon posts within Hugo, similar to what I did for Twitter posts.
With the static embedding, I wanted to remove all requirements for JavaScript, and remove all third-party requests on behalf of the users here. In short, this speeds things up, and saves you from worrying about cookies & tracking, or needing to make exceptions to the site’s CSP.
It’s been a while since I initially looked into what it takes to make an optimal wifi connection with an Espressif ESP8266 / ESP-01 (old post). Using the code from the old post didn’t work … anymore? What the heck. So much for rolling out another kind of battery-powered ESP8266 device. Time to dig into what goes on with the wifi connection timing.
The code is on Github. Most of this post is from the readme file there. Which will rank better in Google, a subdirectory of a well-known domain, or my blog post? Whatever, it’s all the same to me :-).
I wanted to do a short post on how to do something with the Search Console API with R. Backing up a bit, I thought I’d include a short summary of how to get started with R, and as you do, I’m now writing a separate post on how to do that. No, I won’t back up more and explain computers or how to connect your printer.
For today’s excursion we’ll use the programming language “R”. Why do they call it “R”? What happened to all the languages between “C” and “R”? Moving on. But why R? You could do the same things with Python, or most programming languages, but Python is for hipsters, and R is for real data scientists.
I don’t know many (or any) people who think of characters and words in binary, but that might just be a limitation on my end. If you prefer binary, maybe this is something for you. If you just want to geek out, you’re welcome to stay here too.
Using a 2-button macro-keyboard and some Arduino code, you can tap in binary and have the keyboard enter ASCII characters.
I have -too many, err- some extra ESP32 boards so I thought it would be fun to make a small weather display for the SEO weather. The obvious choice is to use MozCast. This is Moz’s unique interpretation of how Google’s search results change. This doesn’t reflect what Google does in its algorithms, but it’s a number for the weather, so why not.
It’s hopefully obvious that this does not represent a recommendation of any particular SEO tool. To be honest, I’m not a fan of any SEO weather reports, in particular because of the way the data is collected. The folks from Moz have been smart & friendly in the past, so I hope they’re OK with this hack. (I’m happy to send someone from Moz one of these, if you drop me a note.)