Search
37 results for “digitalpebble”
-
Missing coding during the end of year break? Want to start the new year with something meaningful?
Why don't you have a go at a this nice #goodfirstissue for SPRUCE :
https://github.com/DigitalPebble/spruce/issues/109
SPRUCE is an #opensource platform for #GreenOps which helps measure and reduce the environmental impact of CloudComputing.
-
New version of SPRUCE available and it's a beauty!
+ Support for split cost allocation data
+ New module for CCF implementation of GPUs
+ Added new generation of GPUs not supported by CCF
+ Bugfix for Boavizta modulehttps://github.com/DigitalPebble/spruce/milestone/8?closed=1
Get it while it's hot!
-
SPRUCE 0.5 is out!
SPRUCE is an open-source tool developed to help organisations measure their cloud-related carbon footprint. It is an enrichment pipeline that processes usage reports generated by AWS and adds environmental impact data, such as the energy used and co2 emissions.
https://github.com/DigitalPebble/spruce/releases/tag/0.5
-
We are pleased to announce the very first release of Spruce
https://github.com/DigitalPebble/spruce
Spruce is an #opensource project which helps estimate the environmental impact of your cloud usage. By leveraging open source models and data, it enriches usage reports generated by AWS and allows you to build reports and visualisations. Having the #greenops and #finops data in the same place makes it easier to expose your costs and impacts side by side.
Contributions, feedback, and questions are welcome!
-
Still early days but making good progress on a brand new #opensource project - Carbonara (might change the name as there already is a project with it).
It does partly what CloudCarbonFootprint used to do i.e. help you measure and reduce the environmental impact from your cloud usage.
-
#stormcrawler has now left the DigitalPebble organisation on GitHub on its way to Apache.
I am not going to lie, I am feeling something: I created SC more than 10 years ago and it became the focal point of my professional activities since.
I am also 100% convinced that it is the right move, at the right time and hope that being an ASF project will help its adoption by users and new contributors -
-
Meet the #StormCrawler users is back on our blog! We are delighted to share this Q&A with members of the OpenWebSearch.eu team. Come and read about their project and how they use both #StormCrawler and #URLFrontier to help deliver a truly open, transparent and legally compliant alternative to the big search engines.
#opensource #openwebsearch #opendata #innovation
https://digitalpebble.blogspot.com/2023/11/meet-stormcrawler-users-q-with-open-web.html
-
#StormCrawler 2.10 is out!
https://github.com/DigitalPebble/storm-crawler/releases/tag/2.10
We have also written a short blog detailing the improvements to the protocol implementations
https://digitalpebble.blogspot.com/2023/10/focus-on-protocol-improvements-in.html
-
@digitalpebble sadly homegrown one off: https://github.com/tballison/file-observatory/tree/main/commoncrawl-fetcher
If I were to do it again, I’d use #ApacheNutch or #StormCrawler
-
We are pleased to announce that DigitalPebble Ltd is a partner of the OpenSearch Project.
In case you have missed it, #StormCrawler has a module for #OpenSearch since its latest release and hopefully there will be more good things to come!
-
Just committed a Maven #archetype for crawling with the #OpenSearch module of #StormCrawler.
-
A very nice contribution to #StormCrawler improving the generation of #WARC files
-
#StormCrawler 2.6 released
https://github.com/DigitalPebble/storm-crawler/releases/tag/2.6
Thanks to our contributors and users
-
Fancy trying the new version of the #StormCrawler archetype which uses #URLFrontier as a backend?
-
SPRUCE 0.9 is out
Here's what's new:
💧 Water is now a first-class metric. SPRUCE can now estimate water consumption — factoring in regional hydric stress.
⚡ Operational emissions got more accurate, with power supply and transmission overheads now included.
📊 A Python-based report generator makes it easier to surface insights from your data.
🚀 and many more improvements under the hood
-
#GreenOps isn't just about carbon — water matters too. 💧
Data centers consume enormous amounts of water, and AI is intensifying that demand. Yet most cloud dashboards (and even some commercial #greenops tools) don't measure it at all.
We just merged a PR in #SPRUCE to add water consumption estimates — including regional water stress data from WRI Aqueduct. A litre used in Canada ≠ a litre used in South Africa.
As far as we know, no other #greenops solution does this. We think it should.
-
#GreenOps isn't just about carbon — water matters too. 💧
Data centers consume enormous amounts of water, and AI is intensifying that demand. Yet most cloud dashboards (and even some commercial #greenops tools) don't measure it at all.
We just merged a PR in #SPRUCE to add water consumption estimates — including regional water stress data from WRI Aqueduct. A litre used in Canada ≠ a litre used in South Africa.
As far as we know, no other #greenops solution does this. We think it should.
-
What truly matters—at work and in life—is getting recognition from those we respect and admire.
We’ve been fortunate to receive several such affirmations recently, and this one from Sopht means a great deal to us.
#greenops #frenchtech #sustainability
PS: there will be a few announcements in the next week or so. Watch this space!
-
A very kind testimonial of our skills in #greensoftware and #greenops from our friends at Tailpipe.ai
-
@jhy thanks! I'll give it a try and upgrade to it before releasing the next version of #StormCrawler
-
Really proud to see both #stormcrawler and #URLFrontier used by OWLer
#OSSYM23 -
If you use #StormCrawler, it makes you an #ApacheStorm user.
Please help the project by filling the survey on https://terminplaner4.dfn.de/EYNJzD9U64UFGOGq -
@tallison @OpenSearchProject
Or just use #StormCrawler ? :mastoinnocent: -
Have added test coverage for #StormCrawler
https://coveralls.io/github/DigitalPebble/storm-crawler?branch=master
As expected pretty low on average, partly explained by the fact that writing tests for Bolts is not trivial but at least we can now see where new tests should be added.
BTW #tests are great #opensource #contributions
-
Hoping to benchmark #StormCrawler + #opensearch with segment replication
-
@davidshq
Definitely. To give an example, one of the top EU online retailers use #StormCrawler but won't publicise (or sponsor) it. Their legal department advised them not to because it would expose the way they use it and that is seen as a risk. -
We're super excited about #StormCrawler being used by the #OpenWebSearch project.
-
Should we support tracing in #stormcrawler? Anyone using tools like Datadog when crawling to track slow URLs and bottlenecks?