Weeknotes

1 files changed, 0 insertions, 196 deletions

diff --git a/served/words/atom.xml b/served/words/atom.xml
deleted file mode 100644
index d8a38d5..0000000
--- a/served/words/atom.xml
+++ /dev/null
@@ -1,196 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<feed xmlns="http://www.w3.org/2005/Atom">
-
-	<title>gennyble's writing</title>
-	<subtitle>Technical writing; project updates; weeknotes</subtitle>
-	<updated>2024-03-02T01:42:00-06:00</updated>
-
-	<link rel="self" href="https://nyble.dev/words/atom.xml" type="application/atom+xml" />
-	<id>https://nyble.dev/words/atom.xml</id>
-
-	<author>
-			<name>gennyble</name>
-			<email>gen@nyble.dev</email>
-	</author>
-
-	
-	<entry>
-		<title>Akkoma Postgres Migration</title>
-		<link href="https://nyble.dev/words/akkoma-postgres-migration.html" rel="alternate" type="text/html" />
-		<id>https://nyble.dev/atom/writing-1/akkoma-postgres-migration</id>
-
-		<published>2023-10-18T23:16:00-05:00</published>
-		<updated>2023-10-18T23:16:00-05:00</updated>
-
-		<content type="html">
-&lt;p&gt;
-&lt;i&gt;(i&apos;m going to say Pleroma a lot here where Akkoma might
-	be correct for newly installed software, but my instance is
-	a few years old and this is more of a telling-of-events than
-	a guide)&lt;/i&gt;
-&lt;/p&gt;
-&lt;details class=&quot;tldr&quot;&gt;
-	&lt;summary&gt;TL;DR; if you migrated your Akkoma&apos;s postgres and now you&apos;re getting timeouts&lt;/summary&gt;
-	&lt;p&gt;
-		It might need a reindex. Use &lt;code&gt;psql&lt;/code&gt; to connect
-		to the database and run &lt;code&gt;REINDEX DATABASE akkoma;&lt;/code&gt;.
-		This might take awhile.
-	&lt;/p&gt;
-&lt;/details&gt;
-&lt;hr/&gt;
-&lt;p&gt;
-Recently I went about trying to get the services running on
-my VPS to be happy in a gig of RAM. I did not achieve this,
-but I found a solution that worked nearly as well.
-&lt;/p&gt;
-&lt;p&gt;
-I wanted to try to scale my VPS, on the &quot;Linode 4GB&quot; plan, back down to a Nanode. It
-started it&apos;s life as a Nanode but Akkoma - well, Pleroma then -
-was greatly displeased with this and pegged my CPU at 100%. Since
-my CPU usage lately peaks at 30% and averages 18%, this no longer
-seems to be the case.
-&lt;/p&gt;
-&lt;p&gt;
-To re-nanode, I had to fit in 1G of memory.
-I managed to shave the 110M I needed
-by asking &lt;code&gt;systemd-journald&lt;/code&gt; to stop using 80M of memory
-&lt;i&gt;(it seemed to ignore my 10M plea, but it dropped by 30M so whatever)&lt;/i&gt;,
-telling Postgres to max use 100M, and disabling things that
-I as not actively using anymore.
-&lt;/p&gt;
-&lt;p&gt;
-I didn&apos;t specifically want to learn the ins-and-outs of Postgres
-performance tuning, so I used &lt;a href=&quot;https://pgtune.leopard.in.ua/&quot;&gt;pgtune&lt;/a&gt;
-to give me the right config lines for 100M. It worked well!
-&lt;/p&gt;
-&lt;p&gt;
-This was all for naught, though, because I couldn&apos;t get my
-disk to fit under 25G, which was also a requirement of nanodeisation that I&apos;d
-forgotten about. The database itself was 9.9G! You can
-&lt;a href=&quot;https://docs.akkoma.dev/stable/administration/CLI_tasks/database/#prune-old-remote-posts-from-the-database&quot;&gt;Prune old remote posts&lt;/a&gt;
-but I didn&apos;t really want to do that yet. It seems like the right
-way to go, but I had one more trick.
-&lt;/p&gt;
-&lt;h2 id=&quot;two-of-them&quot;&gt;Two of Them?&lt;/h2&gt;
-&lt;p&gt;
-I have to keep a separate VPS around for another thing, and it gets
-half a percent of CPU usage, which is... not a lot. All it does is serve
-a single-page static site through Nginx. I could almost
-certainly put this on the same server as all my things, but
-I like having the separation.
-&lt;/p&gt;
-&lt;p&gt;
-This does mean that I pay for almost an entire Nanode to do
-very nearly nothing. 
-&lt;/p&gt;
-&lt;p&gt;
-By putting Postgres on it I&apos;d lose the different-machine aspect
-of the separation, but gain so much disk space and memory. The
-single-page-static is still on a separate public IP which is
-good enough for me!
-&lt;/p&gt;
-&lt;h3 id=&quot;setup-postgres&quot;&gt;Postgres Migration&lt;/h3&gt;
-&lt;i&gt;(more of a recount of events than a guide, but written guidlike? just pay mind to the commands and you&apos;ll be fine)&lt;/i&gt;
-&lt;p&gt;
-Install Postgres on the new server. It doesn&apos;t have to be the
-same major version since we&apos;re going to dump and restore the
-database which is
-&lt;a href=&quot;https://www.postgresql.org/docs/current/upgrading.html&quot;&gt;the recommended upgrade method anyway&lt;/a&gt;.
-Don&apos;t forget to run &lt;code&gt;initdb&lt;/code&gt; and give your data
-directory with the &lt;code&gt;-D&lt;/code&gt; flag. Run it under the
-postgres user.
-&lt;/p&gt;
-&lt;p&gt;
-Now create the database and role that you&apos;ll use. In my experience
-these have to match the database you&apos;re migrating from. I followed
-the &lt;a href=&quot;https://docs.akkoma.dev/stable/administration/backup/#restoremove&quot;&gt;Akkoma database restore/move&lt;/a&gt;
-docs and ended up using psql, again under the postgres user, to run
-&lt;code&gt;CREATE USER akkoma WITH ENCRYPTED PASSWORD &apos;&amp;lt;database-password&amp;gt;&apos;;&lt;/code&gt; and
-&lt;code&gt;CREATE DATABASE akkoma OWNER akkoma;&lt;/code&gt;. &lt;i&gt;(well, i replaced akkoma with pleroma and later used alter queries to change them, but that&apos;s because my database is old)&lt;/i&gt;
-&lt;/p&gt;
-&lt;p&gt;
-After that was ready, I used my firewall of choice (ufw) to
-allow the servers to talk using their private IPs &lt;i&gt;(yay same datacenter)&lt;/i&gt;. After that was done, I ran
-this command &lt;code&gt;pg_dump -U akkoma -C akkoma | ssh dynamo &quot;sudo psql -U akkoma -d akkoma&quot;&lt;/code&gt;
-and waited.
-&lt;i&gt;dynamo&lt;/i&gt; being the host of the new postgres server and owner of a spot in my .ssh/config.
-&lt;/p&gt;
-&lt;p&gt;
-A Note:&lt;br/&gt;
-you can directly do &lt;code&gt;pg_dump ... | psql ...&lt;/code&gt; but the Postgres upgrade
-docs say you need to use the new psql version to upgrade, and the old server was missing that
-binary. Instead of seeing if psql 13 would work or if I could get psql 15 working there, I
-pipped it over ssh.
-&lt;/p&gt;
-&lt;p&gt;
-It completed quicker than I thought, the command only took 21 minutes!, and all seemed well.
-&lt;/p&gt;
-&lt;h3 id=&quot;all-was-not-well&quot;&gt;All Was Not Well&lt;/h3&gt;
-&lt;p&gt;
-First, to prevent Akkoma from receiving activites that may
-be lost if I have to revert, I disallowed everything on 80/443
-except to my own IP so I could see if the web interface was working.
-Yeah my website&apos;d be down for a bit but it was whatever. &lt;i&gt;(i think i could&apos;ve
-	edited the nginx config to the same effect, but this was easier)&lt;/i&gt;
-&lt;/p&gt;
-&lt;p&gt;
-I edited my &lt;code&gt;/etc/pleroma/config.exs&lt;/code&gt; to point
-to the new postgres server and started Akkoma, but new-Postgres didn&apos;t
-see a connection? Oh, I edited the wrong config and it was still
-connecting to the local Postgres.
-&lt;/p&gt;
-&lt;p&gt;
-I deleted &lt;code&gt;/etc/pleroma&lt;/code&gt;, so I&apos;d stop getting confused by
-it, and edited the &lt;i&gt;correct&lt;/i&gt; file: &lt;code&gt;/opt/pleroma/config/prod.secret.exs&lt;/code&gt;
-&lt;i&gt;(this is because I&apos;m a From Source install)&lt;/i&gt;.
-&lt;/p&gt;
-&lt;p&gt;
-Aaaand it didn&apos;t work. Turns out it was trying to connect to it&apos;s own private IP
-because copy-paste can be hard sometimes. Glad I stopped old-Postgres.
-&lt;/p&gt;
-&lt;p&gt;
-Fixing that, I finally saw connections on the other machine. New problem: Akkoma
-timesout the query after 15000ms (15 seconds) because it was taking too long. what?
-and nothing is loading? ahhh.
-&lt;/p&gt;
-&lt;p&gt;
-per the Akkoma docs from earlier, I ran some commands to try and cleanup
-the database. I&apos;m a
-From Source install, so I can &lt;code&gt;mix pleroma.database vacuum analyze&lt;/code&gt;
-which did &lt;i&gt;not help&lt;/i&gt; so I tried it again with &lt;code&gt;full&lt;/code&gt; instead
-of &lt;code&gt;analyze&lt;/code&gt;. This also did not help.
-&lt;/p&gt;
-&lt;p&gt;
-I think what I was looking for was Akkoma to throw a fit as evidence that
-something weird happened during the transfer, but nothing went wrong.
-&lt;/p&gt;
-&lt;p&gt;
-So I was out of ideas. I am a Postgres novice and I&apos;m out of luck. What
-does someone like me do when out of luck? Past the error into Google of course!
-Maybe I should&apos;ve done that from the start, right, but I don&apos;t get
-many results for Akkoma or Pleroma normally.
-&lt;/p&gt;
-&lt;p&gt;
-So to google I went! And pasted &lt;q&gt;timed out because it queued and checked out the connection for longer than 15000ms&lt;/q&gt;
-&lt;/p&gt;
-&lt;p&gt;
-and then I read
-&lt;a href=&quot;https://elixirforum.com/t/timed-out-because-it-queued-and-checked-out-the-connection-for-longer-than-15000ms/34793/4&quot;&gt;a comment from al2o3cr&lt;/a&gt; that said:
-&lt;/p&gt;
-&lt;blockquote&gt;
-	&lt;p&gt;Usually that&apos;s an indication of database issues, from missing indexes to queries that need optimization.&lt;/p&gt;
-&lt;/blockquote&gt;
-&lt;p&gt;
-&quot;Missing indexes&quot; there caught my eye. It made a lot of sense to me. It&apos;s
-taking so long because it&apos;s either digging through the 2.5 million activities
-in the database, or it&apos;s trying to reindex the thing &lt;i&gt;(both?)&lt;/i&gt;. A quick
-google later and I ran &lt;code&gt;REINDEX akkoma;&lt;/code&gt; from psql which literally
-fixed all of my problems.
-&lt;/p&gt;
-&lt;p&gt;
-That&apos;s it! take care and don&apos;t forget to reindex after your migration.
-&lt;/p&gt;
-		</content>
-	</entry>
-	
-</feed>
\ No newline at end of file