Gordon Inggs
-
Finally, *Senior Professional Officer: DevOps Engineer* (*Senior #DevOps Engineer*)
Like many data teams, we own our specialised data infrastructure. This includes @airflow, and #Minio (along with #trino for querying), as well as interactive platforms such as @ProjectJupyter and @posit_glimpse. We run a cloud native, open source stack, with @k3sio for our on-prem setup, and have a growing cloud presence.
Come help keep the magic happening!
Apply at www.capetown.gov.za/careers
6/6
-
Next, *Head: Product Services*
We've been heavily influenced by people like @richardjpope, @pahlkadot, the #GDS, etc. - not having #ProductManagers is why a lot of #govtech doesn't work. We're fixing that by hiring the right people.
We understand good product management to be about making sure that we're building what is needed. This is hard with #data, where there is so much hype and misunderstanding! Come help us do better
Apply at www.capetown.gov.za/careers
3/6
-
*The purpose of a system is what it does*
Please share with anyone that needs this, or might know those that do #CapeTown #SouthAfrica #Observatory #Woodstock #Mowbray
-
Are you a #DataEngineer interested in getting into #CivicTech, but want to still keep a toe-hold in acdaemia?
Our frequent collaborators at #JPAL Africa have a post for a data engineer that will embedded in the City of #CapeTown's Data Science team: https://www.povertyactionlab.org/careers/data-scientist-j-pal-africa-job-105296
Feel free to reach out with any questions
(please boost for reach)
-
Need to do some load testing of an internal HTTP service, mostly emulating interactive workloads. Primarily interested in measuring latency and throughput. Any recommendations?
-
Very cool map built by one of my colleagues on top of the API behind @coct_service_alerts - https://ttlx.me/coct/service-alerts/
-
Not quite the meaning of delivery I had in mind while setting up this news alert.... #CapeTown #ServiceDelivery #gov
-
Well, it seems most of #Bloemfontein is without power. Is giving me a good opportunity to appreciate the new UI of @eskomsepush!
-
CW: PyConZA thread 12/n
Annnddddd finally, @czue finished things off in grand style, talking about his experiences of generating passive income. It served as a really great #IndieHacking 101!
The Tao seemed to be about building simple, effective apps, with a focus on quick iterations, bootstrapping, and testing your product early. Something I really liked was how the real goal of indie hacking isn't necessarily great wealth, but freedom and creative agency! 💻🌱
-
CW: PyConZA thread 11/n
I was then busy as the organisers ill-advisedly gave me a slot, talking about *You don't need a data service, you just need an object store and some JSON files*.
Basically, you can eschew a lot of the complexity of building a service behind an API, if you can actually just get away serving up files from an object store like #Minio, #S3 or #AzureBlobStorage. Putting a CDN in front then makes it really scaleable and cheap!Slides here: https://talks.cct-datascience.xyz/pyconza2024-simple-apis/pyconza2024-simple-apis.slides.html
-
CW: PyConZA thread 9/n
Next up, Johan Beyers (in his own words, *has seen some stuff*) shared how #Takealot use @ApacheKafka for Change Data Capture, Event Sourcing and Task Queues. Definitely gave me a lot to think about, in terms of how data flows across distributed systems. He also had lots of great content on designing components like messages, topics, producers and consumers.
Sounds like a good choice for handling large-scale, high-throughput environments! 🚀 #Kafka #DataEngineering #Scalability
-
Had an excellent two days at #UVUAfrica in Woodstock, #CapeTown.
Yesterday we launched Africa's first City #DataStrategy, and today we had a great workshop led by some excellent experts from orgs such as #OpenCitiesLab, #GDS, #ODI and #eGovFoundation
Check it out - https://resource.capetown.gov.za/documentcentre/Documents/City%20strategies%2c%20plans%20and%20frameworks/CCT_Data_Strategy.pdf
Also, it's just nice to be in a room with my colleagues.
-
A workflow I tried out, with reasonable success, was using my shiny new Kindle Scribe to take notes at #dataconf, then used our Llama 3.1 model to clean up the inevitably mangled OCR output, and organise my thoughts into a series of pithy posts.
Results in this thread: https://elk.zone/fosstodon.org/@Gord1i/113044142959559721
The results weren't magic, as I still had to proof read, and adjust to my tone, etc. but definitely got me a good 60% of the way there.
-
CW: Dataconf 2024 Thread
Finally, @gbelrose continued his theme from last year of applying good software engineering to data engineering, and now reporting. This was pretty inspiring, as it's very tempting to throw your hands up when you encounter a mess of dashboards created via elaborate clicky-clicky.
G advocated using platform APIs to extend your data pipelines into this domain, introspecting and annotate dashboards.
-
CW: Dataconf 2024 Thread
Another great talk was from Zintle Faltein-Maqubela on her experiences running data-focused internship and graduate programmes. Was well timed, as we're hopefully going to be taking on more juniors soonish!
The big point I got was around managing expectations on both sides - particularly for interns, they've often got other things going on in their lives. So getting some understanding of that, against what their team needs from them.
-
CW: Dataconf 2024 Thread
Also, really enjoyed Gerhard van Deventer from #EqualExperts on *Lessons Learned from Using an LLM and Expecting Magic*.
Solid coverage of fundamentals that really matched our experience - Improve your prompting; eval early, eval often; break big tasks into small ones -> set up a flywheel, and start eeking out those iterative improvements.
-
CW: Dataconf 2024 Thread
Over and above James's cool use of interactive polls in his keynote, there were many great points about how to be part of the "community". Something that was key was recognising what you take vs what you give back varies over your career.
And of course, he ended up with a rousing rendition of You Must Check Accessibility (YMCA).
-
CW: Dataconf 2024 Thread
-
Have a PR out for the spec for the data API that backs @coct_service_alerts - https://github.com/cityofcapetown/service-alerts-connector/pull/1
The intent here is to capture the current situation, before following on with a new PR for the `v2` API paths.
Comments welcome!
-
Recently listened through #MichaelLewis's #GoingInfinite in pretty much one sitting while driving cross country.
As someone with a passing interest in cryptocurrencies, effective altruism and probabilistic thinking, it was pretty crazy to hear what happens when it's all turned up to 11.
A bit conflicted about how mental health is mentioned but not directly addressed in the book. It was probably responsible to not armchair diagnose, but it was a big piece of the puzzle to leave out.
-
CW: Dataconf thread
-
CW: Dataconf thread
Next up - Johanna Thathaisa on Low-Code Data Tools!
-
CW: Dataconf thread
Actually, is turning into an enjoying take down of the large consulting world.
Strategy as a service model:
1. Make overblown promises, driving ill-advised adoption of tech (e.g. big data, AI, LLMs)
2. Learn from other people's inevitable failures
3. Sell superior version of (1) -
@danie10 Ja, #OpenUp have done excellent work there, as well as with Vulekamali, National Treasury's more general portal (https://vulekamali.gov.za/) - JD did a great talk on building it at last year's PyConZA - https://www.youtube.com/live/JSfg-H5XIHI?si=Ms0LELiPa0vfgwbu
#OpenCitiesLabs have also done some good stuff, particularly with narrative content on the Durban Edge portal - https://edge.durban/
These are all for dissemination though - I would like to push the envelope, along the line of O'Reilly's gov-as-a-platform idea.
-
Getting into @pahlkadot's Recoding America (which is amazing) and it's eerie how relevant it is to my experience in local gov on the other side of the world.
> Elites understand policy. They are comfortable with it. They may also be comfortable with politics, which is often described as "how the policy gets done." The rest of us know about delivery.