#dag — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #dag, aggregated by home.social.
-
Как я превратила Obsidian в структурированную память для ИИ‑агентов
Эта статья про NOUZ — локальный MCP‑сервер между Obsidian и ИИ‑агентом. Он превращает базу заметок в структурированную память: с уровнями, связями и сигналами дрейфа. Внутри — как я пришла к этой архитектуре и что она даёт агенту при работе с базой.
-
Как я превратила Obsidian в структурированную память для ИИ‑агентов
Эта статья про NOUZ — локальный MCP‑сервер между Obsidian и ИИ‑агентом. Он превращает базу заметок в структурированную память: с уровнями, связями и сигналами дрейфа. Внутри — как я пришла к этой архитектуре и что она даёт агенту при работе с базой.
-
𝗘𝗲𝗿𝘀𝘁𝗲 𝘇𝗼𝗺𝗲𝗿𝘀𝗲 𝗱𝗮𝗴 𝗶𝘀 𝗲𝗲𝗻 𝗳𝗲𝗶𝘁: 25,1 𝗴𝗿𝗮𝗱𝗲𝗻 𝗶𝗻 𝗘𝗹𝗹
In het Limburgse Ell werd het vandaag 25,1 graden, waardoor het officieel de eerste lokale zomerse dag is. Van een officiële zomerse dag is nu nog geen sprake, want in De Bilt is het nog geen 25 graden. Morgen kan het 26 graden worden, later volgen lokaal fikse onweersbuien.
https://www.rtl.nl/nieuws/weer/artikel/5596840/eerste-zomerse-dag-een-feit-251-graden-ell
-
Directed Acyclic Graphs (DAGs) and Regression for Causal Inference
by Susan Alber (2022) https://health.ucdavis.edu/media-resources/ctsc/documents/pdfs/directed-acyclic-graphs20220209.pdf#intervention #policy #causality #causalInference #stats #statistics #counterFactuals #probabilities #causation #confounding #DAG #DAGs #graphs
-
Directed Acyclic Graphs (DAGs) and Regression for Causal Inference
by Susan Alber (2022) https://health.ucdavis.edu/media-resources/ctsc/documents/pdfs/directed-acyclic-graphs20220209.pdf#intervention #policy #causality #causalInference #stats #statistics #counterFactuals #probabilities #causation #confounding #DAG #DAGs #graphs
-
Directed Acyclic Graphs (DAGs) and Regression for Causal Inference
by Susan Alber (2022) https://health.ucdavis.edu/media-resources/ctsc/documents/pdfs/directed-acyclic-graphs20220209.pdf#intervention #policy #causality #causalInference #stats #statistics #counterFactuals #probabilities #causation #confounding #DAG #DAGs #graphs
-
Directed Acyclic Graphs (DAGs) and Regression for Causal Inference
by Susan Alber (2022) https://health.ucdavis.edu/media-resources/ctsc/documents/pdfs/directed-acyclic-graphs20220209.pdf#intervention #policy #causality #causalInference #stats #statistics #counterFactuals #probabilities #causation #confounding #DAG #DAGs #graphs
-
Directed Acyclic Graphs (DAGs) and Regression for Causal Inference
by Susan Alber (2022) https://health.ucdavis.edu/media-resources/ctsc/documents/pdfs/directed-acyclic-graphs20220209.pdf#intervention #policy #causality #causalInference #stats #statistics #counterFactuals #probabilities #causation #confounding #DAG #DAGs #graphs
-
She is bad, #Blanche is far worse.
#Trump fired AG #PamBondi, acc/to a person familiar, removing the nation’s top law enforcement officer as his frustration with her job performance deepened. Trump has been souring on Bondi for months, especially because of her handling of the #EpsteinFiles, which has become a political liability for Trump among his supporters. #ToddBlanche, the deputy attorney general [#DAG], will be acting AG, the person said.
#law #LegalEthics #DOJ
https://www.cnn.com/2026/04/02/politics/pam-bondi-role-trump?cid=ios_app -
Screenshot as I'm visually validating a (TINY) portion of our model of an irrigation district's water distribution network with a bespoke viewer made with Cytoscape.
If you work with #Graph, #Network, #DAG, or graph structures in general, #Cytoscape is a wonderful library to work with. It has lots of plugins, layouts, and examples.
-
Aviation weather for Barstow Daggett airport (USA) is “KDAG 301950Z AUTO 00000KT 10SM CLR 17/M06 A3037 RMK AO2 SLP278 T01721056” : See what it means on https://www.bigorre.org/aero/meteo/kdag/en #barstowdaggettairport #airport #daggett #usa #kdag #dag #metar #aviation #aviationweather #avgeek vl
-
𝗗𝗮𝗴 𝗯𝗲𝗴𝗶𝗻𝘁 𝗯𝗲𝘄𝗼𝗹𝗸𝘁, 𝗹𝗮𝘁𝗲𝗿 𝘀𝘁𝗲𝗲𝗱𝘀 𝗺𝗲𝗲𝗿 𝗼𝗽𝗸𝗹𝗮𝗿𝗶𝗻𝗴𝗲𝗻 𝗲𝗻 𝘇𝗼𝗻
Na het grijze weer van gisteren komt vandaag de zon er weer eens door. Toch blijven er nog steeds ook wolkenvelden over en kan heel lokaal een spatje regen vallen. Het wordt tussen 0 en 7 graden.
-
Spent the afternoon chopping down neighbour’s gigantic plum tree that had fallen victim to the high winds. Real shame.
Got a nice bucket of plums out of it.
Also, looks like I got back some of the hemp fibres from my weed mats in the form of this nest.
-
𝗡𝗮 𝗸𝗼𝘂𝗱𝗲 𝗻𝗮𝗰𝗵𝘁 𝗲𝗲𝗻 𝘇𝗼𝗻𝗻𝗶𝗴𝗲 𝗱𝗮𝗴: 𝗹𝗶𝗰𝗵𝘁𝗲 𝘃𝗼𝗿𝘀𝘁 𝗲𝗻 𝗹𝗮𝘁𝗲𝗿 𝗺𝗲𝗲𝗿 𝘄𝗼𝗹𝗸𝗲𝗻
Na een heldere nacht kan het op veel plaatsen licht vriezen. Lokaal daalt de temperatuur tot -4 graden. Overdag schijnt de zon volop, al trekken er later ook sluierwolken over het land. Het wordt uiteindelijk 5 tot 9 graden.
https://www.rtl.nl/nieuws/weer/artikel/5556699/weerbericht-20-januari-2026
-
While an #FBI probe is “ongoing”, lawyers in the #CivilRights Division were informed last week that they would not play a role in the investigation at this time, acc/to 2 people….
And on Tuesday, #DAG [& douche] #ToddBlanche said in a statement that “there is currently no basis for a criminal civil rights investigation.” The statement, first reported by CNN, did not elaborate on how the #DOJ had reached a conclusion that no investigation was warranted.
-
On April 30, Singh said in an email to McGuire that the prosecution was a “top priority” for the #DAG’s Office, according to the order.
[just like in Trump’s BS election fraud lawsuits, I suppose we ought to be grateful that these stooges are such shit at committing their crimes. If they were actually capable, they might get away with it all]
#Trump #law #LegalEthics #immigration #KilmarAbregoGarcia #kakistocracy #DOJ #CoverUp
-
https://www.europesays.com/it/268602/ Ecco la Digital Art Gallery, così Gorizia diventa «Capitale dell’arte digitale» • Il Goriziano #anzil #Arte #ArteEDesign #ArteEdesign #Arts #ArtsAndDesign #callari #dag #Design #DigitalArtGallery #Entertainment #fedriga #GalleriaBombi #Intrattenimento #IT #Italia #Italy #ledwall #SamoTurel #ziberna
-
Чистим main.go: предсказуемый старт и надежный Graceful Shutdown
Сталкивались ли вы с болью при управлении порядком запуска и остановки зависимостей в вашем Go-сервисе? Разработка больших сервисов неизбежно приводит к необходимости управлять множеством зависимостей. В этом контексте мы говорим о долгоживущих компонентах , чья работа обеспечивается отдельными горутинами: как правило, это блокирующий метод (например, Start ), внутри которого крутится цикл обработки. Примерный сценарий жизненного цикла сервиса выглядит так: При запуске критически важно, чтобы пул соединений с БД, кэш и очереди были полностью готовы до того, как HTTP-сервер откроет порт и начнет принимать входящий трафик. С graceful shutdown ситуация обратная: порядок должен быть строго зеркальным. Сначала нужно перестать принимать новые запросы, дождаться завершения текущих, остановить воркеры, и только потом разрывать соединения с инфраструктурой. Иначе мы получаем неприятные ошибки подключения и даже потерянные транзакции в момент деплоя. Если эти проблемы вам не знакомы, смело закрывайте вкладку. Скорее всего, эта статья не принесет вам пользы. Но если вы ищете способ автоматизировать эту рутину, сохранив код чистым - добро пожаловать под кат.
https://habr.com/ru/articles/976800/
#go #golang #graceful_shutdown #dag #Dependency_Injection #Uber_Fx #Микросервисы #Open_Source #Архитектура #lifecycle
-
-
DAG-классификация: как мы научили поиск определять нужную категорию ступенчатым образом
Одна из важнейших задач поиска — релевантная выдача. Простых универсальных решений здесь нет, а улучшение поиска — долгосрочный процесс, где крупные задачи приходится разбивать на небольшие, последовательные шаги. В этой статье делимся тем, как нам в «Магнит Маркете» удалось значительно улучшить качество поиска с помощью нетривиального подхода: ступенчатой классификации категории поискового запроса.
https://habr.com/ru/companies/magnit/articles/975980/
#ml #dag #поиск #ранжирование #релевантный_поиск #релевантность_поисковой_выдачи #data_science #dagмодель #оптимизация_поиска
-
I am thinking about an interesting problem right now.
Suppose I have a DAG of objects. Every object contains three data points:
* A list of parents (via Content-address hashes)
* A pointer to some content (almost irrelevant for this thought process)
* A version number (irrelevant for this thought process)I have a limited but unknown number of peers that are allowed to "post" to that DAG. Once a peer discovers that another node has posted to the DAG, they either fast-foward or merge (which is trivial here) and go on.
All peers gossip all the time, so fast-forwarding is expected to be the "normal case" when all peers are online - but in case of network split there is no issue.Now, suppose I want to allow "rewriting" the DAG.
That means, one node decides that deep down in the DAG, they want to change a node. That would change all other nodes that come after it.How would the other peers know that the node was rewritten?
Two ideas:
* All peers keep track of "this other peer points to this hash right now". Once a peer rewrites their DAG, other peers can see that rather easily. That would involve some tricky logic, but I guess would be possible 🤔 The other peers can then update their stuff to that new DAG (and if needed even "rebase" changes that they have done between the rewrite and now, if there was a network split during that time)
* The second option would involve adding timestamps to the DAG nodes, so other nodes can see that a portion of the DAG was rewrittenThe second option would add more fields to the DAG nodes, which I would like to not do, because they should be as light as possible.
What do you think? :boost_ok:
#algorithms #softwaredevelopment #dag #distributedsystems #graphdata #datatypes
-
This mysterious blob is evidence of a male sparrow caught brown beaked stealing hemp from the hemp weed mat I put on the Rimu.
I noticed last night it was shredded and now I’ve seen the culprit!
-
The inquiry, according to the additional sources, include the office of #DAG [& fmr Trump personal atty] #ToddBlanche at #DOJ headquarters looking into the work of people who may have been dispatched or held themselves out to witnesses improperly around the mortgage fraud investigation.
#law #LegalEthics #LegalProcedure #RevengePolitics #Trump #AdamSchiff #EdMartin #BillPulte
-
Earlier in the day, #DOJ lawyer Tyler Lemons, who is prosecuting the case, also told U.S. District Judge Michael Nachmanoff that “someone” in #DAG #ToddBlanche’s office ordered him not to disclose whether career prosecutors in the Department of Justice authored a memo recommending that Comey should not be indicted. Lemons said that he was told he couldn’t disclose “privileged” information without permission.
#law #LegalEthics #LegalProcedure #RevengePolitics #Trump #JamesComey
-
#DOJ lawyer Tyler Lemons declined to say whether prosecutors have submitted a memo recommending that #JamesComey should not be prosecuted. Lemons told the judge that someone in #DAG #ToddBlanche’s office told him he couldn’t disclose “privileged” information without their permission.
“At this point, my position would be, whether there was a declination memo, is privileged,” Lemons said, adding that “I don’t know in the world of documents there is a declination memo.”
-
-
Here’s Chris Hayes & lawyer / law professor / fmr US attorney / fmr asst #DAG Harry Litman discussing the pretty obvious #CoverUp by DAG #ToddBlanche of #Trump’s involvement with #Epstein during his “questioning” of #GhislaineMaxwell.
#law #Trump #EpsteinFiles #corruption #DOJ
https://www.youtube.com/watch?v=RhRXU-WbKAo -
#Epstein dump exposed #DOJ's massive #CoverUp for #Trump
Law professor & co-editor-in-chief of Just Security Ryan Goodman said the latest documents from #JeffreyEpstein is explosive & seems to implicate Trump's deputy attorney general [#DAG] in a massive cover-up.
The emails released directly contradict statements Epstein's associate, #GhislaineMaxwell, made to DAG #ToddBlanche in a taped interview designed to exonerate Trump.
#law #EpsteinFiles #corruption #DOJ
https://www.rawstory.com/todd-blanche-2674290392/ -
STACD: STAC and DAGs: #STACD, a new extension for SpatioTemporal Asset Catalogs (#STAC) from @spatialnodehq leverages #DAG-s to track data lineage, version algorithms, and efficiently re-compute only affected downstream STAC assets when an upstream asset is updated.
https://spatialists.ch/posts/2025/10/30-stacd-stac-and-dags/ #GIS #GISchat #geospatial #SwissGIS -
𝗔𝗰𝘁𝘂𝗮𝗹𝗶𝘁𝗲𝗶𝘁𝗲𝗻𝗽𝗿𝗼𝗴𝗿𝗮𝗺𝗺𝗮 𝗡𝗶𝗲𝘂𝘄𝘀 𝘃𝗮𝗻 𝗱𝗲 𝗗𝗮𝗴 𝗼𝗼𝗸 𝗶𝗻 2026 𝘁𝗲 𝘇𝗶𝗲𝗻 𝗼𝗽 𝘁𝗲𝗹𝗲𝘃𝗶𝘀𝗶𝗲
Het actualiteitenprogramma 'Nieuws van de Dag' is sinds het begin van het jaar een vaste waarde in de vooravond op SBS6. Talpa heeft nu besloten dat de nieuwsrubriek ook in 2026 iedere werkdag op de buis blijft.
-
FlowSynx – Orchestrate Declarative, Plugin-Driven DAG Workflows on .NET
#HackerNews #FlowSynx #Declarative #Workflows #.NET #DAG #Orchestration #PluginDriven
-
Deml: The Directed Acyclic Graph Elevation Markup Language
https://github.com/Mcmartelle/deml
#HackerNews #Deml #DAG #Elevation #Markup #Language #Graph #Technology #OpenSource
-
CW: Long HiveMind request: Guidelines for causal DAGs in Bayesian modelling
Fediverse #HiveMind request: Guidelines for causal DAGs in Bayesian modelling
One of my many personal peeves is sloppy conceptual diagramming. I have been exposed to far too many box and arrow diagrams where there is no clear and consistent interpretation of what types of things the boxes are supposed to represent (and ditto for the arrows).
I am starting to engage with a Bayesian modelling project (this I know from nothing - https://tomlehrersongs.com/wp-content/uploads/2018/12/lobachevsky.pdf) and am being exposed to causal DAGs. I presume these are reasonably consistent and interpretable. Nonetheless, they are typically presented as though the interpretation is self-evident and I keep coming up with multiple possible interpretations/constraints.
>>>>> Do people have recommendations for *detailed* interpretations of causal DAGs as used in Bayesian modelling? <<<<<
Issues covered should include things like:
* Constraints on node types: If nodes represent variables can they have multivariate values or are they constrained to be univariate?
* Interpretation with respect to "cases": Should the DAG be interpreted as referring to relationships that hold on a per-case basis?
* Interpretation with respect to "populations": If DAGs represent per-case relations, do they also represent populatiopns of those cases?
* Interpretation with respect to data matrices: How does a causal DAG map onto a matrix of data to be modelled?
* Interpretation with respect to multiple types of "case": Repeat all of the issues above but when there are multiple types of case not necessarily in one-to-one relationships (e.g. multi-level modelling).
* What is the relationship between the DAG and the equations/statements implementing that DAG in the probabilistic programming language of choice (e.g. STAN, PyMC)?
* What are the constraints on the temporal relationships of the variables at the nodes? Assuming the variables are measures at points in time you (presumably) don't want causes coming after their effects. Different variables may be measured at different points in time and a measurement at some point in time may be a measure of some cumulative process over prior times - so I would expect some nuanced consideration of temporal constraints.#Bayesian #modelling #statistics #causal #DAG #DirectedAcylicGraph #diagram #interpretation
-
CW: Long HiveMind request: Guidelines for causal DAGs in Bayesian modelling
Fediverse #HiveMind request: Guidelines for causal DAGs in Bayesian modelling
One of my many personal peeves is sloppy conceptual diagramming. I have been exposed to far too many box and arrow diagrams where there is no clear and consistent interpretation of what types of things the boxes are supposed to represent (and ditto for the arrows).
I am starting to engage with a Bayesian modelling project (this I know from nothing - https://tomlehrersongs.com/wp-content/uploads/2018/12/lobachevsky.pdf) and am being exposed to causal DAGs. I presume these are reasonably consistent and interpretable. Nonetheless, they are typically presented as though the interpretation is self-evident and I keep coming up with multiple possible interpretations/constraints.
>>>>> Do people have recommendations for *detailed* interpretations of causal DAGs as used in Bayesian modelling? <<<<<
Issues covered should include things like:
* Constraints on node types: If nodes represent variables can they have multivariate values or are they constrained to be univariate?
* Interpretation with respect to "cases": Should the DAG be interpreted as referring to relationships that hold on a per-case basis?
* Interpretation with respect to "populations": If DAGs represent per-case relations, do they also represent populatiopns of those cases?
* Interpretation with respect to data matrices: How does a causal DAG map onto a matrix of data to be modelled?
* Interpretation with respect to multiple types of "case": Repeat all of the issues above but when there are multiple types of case not necessarily in one-to-one relationships (e.g. multi-level modelling).
* What is the relationship between the DAG and the equations/statements implementing that DAG in the probabilistic programming language of choice (e.g. STAN, PyMC)?
* What are the constraints on the temporal relationships of the variables at the nodes? Assuming the variables are measures at points in time you (presumably) don't want causes coming after their effects. Different variables may be measured at different points in time and a measurement at some point in time may be a measure of some cumulative process over prior times - so I would expect some nuanced consideration of temporal constraints.#Bayesian #modelling #statistics #causal #DAG #DirectedAcylicGraph #diagram #interpretation
-
CW: Long HiveMind request: Guidelines for causal DAGs in Bayesian modelling
Fediverse #HiveMind request: Guidelines for causal DAGs in Bayesian modelling
One of my many personal peeves is sloppy conceptual diagramming. I have been exposed to far too many box and arrow diagrams where there is no clear and consistent interpretation of what types of things the boxes are supposed to represent (and ditto for the arrows).
I am starting to engage with a Bayesian modelling project (this I know from nothing - https://tomlehrersongs.com/wp-content/uploads/2018/12/lobachevsky.pdf) and am being exposed to causal DAGs. I presume these are reasonably consistent and interpretable. Nonetheless, they are typically presented as though the interpretation is self-evident and I keep coming up with multiple possible interpretations/constraints.
>>>>> Do people have recommendations for *detailed* interpretations of causal DAGs as used in Bayesian modelling? <<<<<
Issues covered should include things like:
* Constraints on node types: If nodes represent variables can they have multivariate values or are they constrained to be univariate?
* Interpretation with respect to "cases": Should the DAG be interpreted as referring to relationships that hold on a per-case basis?
* Interpretation with respect to "populations": If DAGs represent per-case relations, do they also represent populatiopns of those cases?
* Interpretation with respect to data matrices: How does a causal DAG map onto a matrix of data to be modelled?
* Interpretation with respect to multiple types of "case": Repeat all of the issues above but when there are multiple types of case not necessarily in one-to-one relationships (e.g. multi-level modelling).
* What is the relationship between the DAG and the equations/statements implementing that DAG in the probabilistic programming language of choice (e.g. STAN, PyMC)?
* What are the constraints on the temporal relationships of the variables at the nodes? Assuming the variables are measures at points in time you (presumably) don't want causes coming after their effects. Different variables may be measured at different points in time and a measurement at some point in time may be a measure of some cumulative process over prior times - so I would expect some nuanced consideration of temporal constraints.#Bayesian #modelling #statistics #causal #DAG #DirectedAcylicGraph #diagram #interpretation
-
CW: Long HiveMind request: Guidelines for causal DAGs in Bayesian modelling
Fediverse #HiveMind request: Guidelines for causal DAGs in Bayesian modelling
One of my many personal peeves is sloppy conceptual diagramming. I have been exposed to far too many box and arrow diagrams where there is no clear and consistent interpretation of what types of things the boxes are supposed to represent (and ditto for the arrows).
I am starting to engage with a Bayesian modelling project (this I know from nothing - https://tomlehrersongs.com/wp-content/uploads/2018/12/lobachevsky.pdf) and am being exposed to causal DAGs. I presume these are reasonably consistent and interpretable. Nonetheless, they are typically presented as though the interpretation is self-evident and I keep coming up with multiple possible interpretations/constraints.
>>>>> Do people have recommendations for *detailed* interpretations of causal DAGs as used in Bayesian modelling? <<<<<
Issues covered should include things like:
* Constraints on node types: If nodes represent variables can they have multivariate values or are they constrained to be univariate?
* Interpretation with respect to "cases": Should the DAG be interpreted as referring to relationships that hold on a per-case basis?
* Interpretation with respect to "populations": If DAGs represent per-case relations, do they also represent populatiopns of those cases?
* Interpretation with respect to data matrices: How does a causal DAG map onto a matrix of data to be modelled?
* Interpretation with respect to multiple types of "case": Repeat all of the issues above but when there are multiple types of case not necessarily in one-to-one relationships (e.g. multi-level modelling).
* What is the relationship between the DAG and the equations/statements implementing that DAG in the probabilistic programming language of choice (e.g. STAN, PyMC)?
* What are the constraints on the temporal relationships of the variables at the nodes? Assuming the variables are measures at points in time you (presumably) don't want causes coming after their effects. Different variables may be measured at different points in time and a measurement at some point in time may be a measure of some cumulative process over prior times - so I would expect some nuanced consideration of temporal constraints.#Bayesian #modelling #statistics #causal #DAG #DirectedAcylicGraph #diagram #interpretation
-
CW: Long HiveMind request: Guidelines for causal DAGs in Bayesian modelling
Fediverse #HiveMind request: Guidelines for causal DAGs in Bayesian modelling
One of my many personal peeves is sloppy conceptual diagramming. I have been exposed to far too many box and arrow diagrams where there is no clear and consistent interpretation of what types of things the boxes are supposed to represent (and ditto for the arrows).
I am starting to engage with a Bayesian modelling project (this I know from nothing - https://tomlehrersongs.com/wp-content/uploads/2018/12/lobachevsky.pdf) and am being exposed to causal DAGs. I presume these are reasonably consistent and interpretable. Nonetheless, they are typically presented as though the interpretation is self-evident and I keep coming up with multiple possible interpretations/constraints.
>>>>> Do people have recommendations for *detailed* interpretations of causal DAGs as used in Bayesian modelling? <<<<<
Issues covered should include things like:
* Constraints on node types: If nodes represent variables can they have multivariate values or are they constrained to be univariate?
* Interpretation with respect to "cases": Should the DAG be interpreted as referring to relationships that hold on a per-case basis?
* Interpretation with respect to "populations": If DAGs represent per-case relations, do they also represent populatiopns of those cases?
* Interpretation with respect to data matrices: How does a causal DAG map onto a matrix of data to be modelled?
* Interpretation with respect to multiple types of "case": Repeat all of the issues above but when there are multiple types of case not necessarily in one-to-one relationships (e.g. multi-level modelling).
* What is the relationship between the DAG and the equations/statements implementing that DAG in the probabilistic programming language of choice (e.g. STAN, PyMC)?
* What are the constraints on the temporal relationships of the variables at the nodes? Assuming the variables are measures at points in time you (presumably) don't want causes coming after their effects. Different variables may be measured at different points in time and a measurement at some point in time may be a measure of some cumulative process over prior times - so I would expect some nuanced consideration of temporal constraints.#Bayesian #modelling #statistics #causal #DAG #DirectedAcylicGraph #diagram #interpretation
-
Основы ETL на примере работы с Superset, Airflow и ClickHouse
В этой статье я расскажу, как можно запустить простой ETL-процесс на виртуальном сервере, используя связку Superset, Airflow и ClickHouse. В качестве платформы я взял готовую конфигурацию от Beget, включающую Superset и Airflow из коробки — это позволяет сосредоточиться на логике обработки данных, а не на настройке окружения. В качестве примера мы подготовим процесс выгрузки и визуализации данных о товарах с сайта Wildberries. Для извлечения данных мы будем использовать Python-библиотеки selenium и BeautifulSoup — они хорошо подходят для парсинга веб-страниц. Дополнительно применим re для обработки текстовой информации с помощью регулярных выражений.
https://habr.com/ru/companies/beget/articles/928712/
#etl #apache_airflow #apache_superset #clickhouse #dag #обработка_данных #biинструменты #анализ_данных #beget
-
𝗘𝗲𝗿𝘀𝘁𝗲 𝗹𝗼𝗸𝗮𝗹𝗲 𝘄𝗮𝗿𝗺𝗲 𝗱𝗮𝗴: 20,0 𝗴𝗿𝗮𝗱𝗲𝗻 𝗶𝗻 𝗗𝗲𝗲𝗹𝗲𝗻
De eerste lokale warme dag van dit jaar is een feit. In het Gelderse Deelen steeg de temperatuur vanmiddag naar 20,0 graden. Voor een warme dag moet het op één van de officiële meetpunten in het land 20 graden of meer worden.
-
#HealthWorkers Plan #GlobalDayOfAction to Demand 'End to the #Genocide in #Gaza' - Monday, January 6th, 2025
"After witnessing 15 months of relentless violence and destruction in Gaza, we can no longer carry on as if everything is normal," said organizer #DoctorsAgainstGenocide.
Brett Wilkins
Jan 03, 2025"As Israel's 15-month annihilation of Gaza continues with intensified attacks on medical infrastructure and workers, an international coalition of advocacy groups is planning a #SickFromGenocide global day of action on Monday 'to take a stand against the targeted attacks on healthcare.'
"Organizer Doctors Against Genocide (#DAG) and co-sponsors including #HealthcareWorkersForPalestine, #PalestinianYouthMovement, #DoNoHarmCoalition, #LaborForPalestine, #JewishVoiceForPeace Health Advisory Council, and others are calling on healthcare workers around the world to take a day of mental health leave 'to reflect on the immense moral injury of funding a genocide and engage the most important aspect of treatment: publicly demanding an end to the genocide in Gaza.'
"Monday's day of action is set to include a '#SickFromGenocide' global vigil and pop-up clinics in cities across the United States, whose government gives Israel billions of dollars in weapons support each year.
"'For 15 months, we have watched in horror as children and families have been obliterated by unrelenting attacks,' DAG said in a statement Friday. 'Hospitals, the bedrock of lifesaving care, have been turned into death traps. The recent bombing and burning of #KamalAdwanHospital and the arrest of our colleague, the pediatrician Dr. #HussamAbuSafiya,exemplify the deliberate targeting of healthcare workers and facilities—tactics designed to accelerate the annihilation and forced displacement of the Palestinian people in Gaza.'"
Read more:
https://www.commondreams.org/news/doctors-against-genocide-protest
#FreePalestine #GazaGenocide #Genocide #BibiIsAWarCriminal #IsraeliWarCrimes #HumanRightsAreNeverWrong #StopArmingIsrael #IDF #GenevaConvention #HumanRightsViolations #NoWar #WorldWarBibi -
Как решения Data Access Governance и Data Centric Audit Protection помогают бороться с утечками данных
По данным центра мониторинга внешних цифровых угроз ГК «Солар», за 2023 год в публичный доступ попали данные почти 400 российских организаций. Это означает, что в прошлом году каждый день в России происходила как минимум одна утечка данных. Основной массив утекших данных – это архивы внутренней документации, в том числе сканы документов, дампы (снимки) памяти систем, информация с компьютеров отдельных пользователей, исходный код программных продуктов. На теневых ресурсах постоянно появляются предложения о продаже подобных данных, причем объем растет от года к году. Так, по данным другого исследования ГК «Солар», в 2023 году число подобных объявлений выросло в сравнении с 2022 годом на 42%, а за первые месяцы 2024 года в даркнет утекли данные 170 компаний — 40% от всего числа инцидентов за 2023 год. Взрывной рост угроз явно подчеркивает важность контроля над корпоративными хранилищами. В этой статье Дмитрий Богомолов, архитектор из команды Центра технологий кибербезопасности ГК "Солар", рассказывает, как системы класса DAG/DCAP помогают организациям взять данные под контроль и оптимизировать работу систем централизованного хранения информации.
https://habr.com/ru/companies/solarsecurity/articles/866422/
#dag #dcap #управление_доступом_к_данным #управление_доступом
-
@cryptonewsbot today the vote has started for #IOTA Rebased. #cryptocurrency #Layer1 #DLT #DAG
-
𝗭𝗼𝗻𝗼𝘃𝗲𝗿𝗴𝗼𝘁𝗲𝗻 𝗱𝗮𝗴 𝘃𝗼𝗼𝗿 𝗱𝗲 𝗯𝗼𝗲𝗴, 𝗻𝗮 𝗲𝗲𝗿𝘀𝘁𝗲 𝗻𝗮𝗰𝗵𝘁 𝗺𝗲𝘁 𝘃𝗼𝗿𝘀𝘁
De afgelopen nacht kwam het in Eelde tot de eerste vorst van het najaar, terwijl we daar vorig jaar aanzienlijk langer op moesten wachten. Toen werd de eerste vorst pas op 12 november gemeten. De komende dagen wordt het echter aanzienlijk warmer, met morgen in het zuiden lokaal 23 tot 24 graden.
https://www.rtl.nl/nieuws/artikel/5475411/zonovergoten-dag-voor-de-boeg-na-eerste-nacht-met-vorst
-
Агрегация данных для аналитики продаж с помощью DataSphere Jobs и Airflow SDK
В маркетинге и продажах крупных компаний есть несколько аналитических задач, которые требуют регулярной обработки сотен тысяч и миллионов записей из разных источников. Например, это прогнозирование продаж или планирование рекламных кампаний. Как правило, их решение не обходится без построения длинного пайплайна обработки данных. ML‑инженеру или аналитику данных нужен ансамбль из нескольких моделей и сервисов, чтобы собрать качественный датасет, провести эксперименты и выбрать наиболее подходящие алгоритмы. Сбор, очистка и агрегация данных занимают большую часть времени и вычислительных ресурсов, а эти затраты хочется оптимизировать. В статье покажем, как мы ускорили построение пайплайнов обработки данных с помощью связки DataSphere Jobs и Apache Airflow™.
https://habr.com/ru/companies/yandex_cloud_and_infra/articles/839494/
-
Построение надёжных систем из ненадёжных агентов
Большие языковые модели можно применять для разных практических целей. Одно из самых интересных направлений — это автономные AI-агенты. Если сгенерировать большое количество агентов по заданному запросу и заставить их конкурировать друг с другом, то теоретически можно получить оптимальный результат по данной проблеме. Это можно использовать и в информационной безопасности, и в других сферах программной разработки. Кроме того, можно создавать агентов, то есть софт, который самостоятельно эволюционирует и улучшает себя на базе обратной связи от пользователей.
https://habr.com/ru/companies/globalsign/articles/822169/
#агенты #overkiLLM #ollama #LLM #DAG #DAGWorks #Burr #AIагенты #надёжность #Retrieval_Augmented_Generation #RAG #Instructor #prompt_engineering #openllmetry #openinference #OpenTelemetry #pgvector #RAGatouille
-
#Elezioni #Turchia #Locali #Smirne
Intention poll di BETİMAR:Cemil #Tugay (#CHP|S&D): 40,9%
Hamza #Dağ (#AKP|Destra): 40,6%
Ümit #Özlale (#İYİ|Centro-destra): 6,6%
Akın #Birdal (#DEM|Sinistra verde): 5,3%
Naşit #Birgüvi (#Zafer|Destra radicale kemalista): 3%
Cemal #Arıkan (#YRP|Estrema destra islamica): 1,5%
Cüneyt #Oğuz (#MP|Centro-sinistra kemalista): 0,8% -
I also need to be researching and writing more about multi-modal #causal #DigitalTwins in these contexts, especially with the advent of liquid neural networks #LNN within the realm of truly #Bayesian neural networks using empirical Bayes and weighted Bayesian variables for a priori similarity of engineering and technical parameters expressed as directed acyclic graphs #DAG within the digital twins of the #SensAE and interactions/dependencies of neighboring sensor analytics ecosystems
-
Fantom wants to cut token burn rate by 75% to fund dApp rewards program - "Fantom will become the youtube/twitch of blockchain platforms," ... - https://cointelegraph.com/news/fantom-wants-to-cut-token-burn-rate-by-75-to-fund-dapp-rewards-program #cryptocurrencies #tokenburn #fantom #dag
