#compaction — Public Fediverse posts

[Перевод] Когда, зачем и как правильно начинать новую сессию в Claude Code?

То, как вы управляете сессиями, контекстом и компактизацией в Claude Code, влияет на результат сильнее, чем кажется. Мы выпустили /usage — новую slash-команду, которая помогает отслеживать использование Claude Code. Эта функция появилась после серии разговоров с пользователями. В этих разговорах раз за разом всплывала одна тема: огромный разброс в том, как люди управляют сессиями — особенно после увеличения контекстного окна до 1 миллиона токенов. Держать одну или две долгосрочные сессии в терминале? Начинать новую с каждым промптом? Когда использовать compact , rewind или subagents ? Что приводит к плохой компактизации или плохой сессии? Всё это неожиданно сильно влияет на опыт работы с Claude Code , и почти всё сводится к управлению контекстным окном .

https://habr.com/ru/articles/1024038/

#Claude_Code #контекстное_окно #управление_сессиями #context_rot #compaction #rewind #subagents #AIагенты #LLMинструменты #Anthropic

Как мы пересобрали сборку мусора в Vinyl

В предыдущей статье о Vinyl я рассказывал об архитектуре LSM-движка Tarantool. Восемь лет, прошедшие с момента с написания статьи, показали, что Vinyl сразу получился идеальным и менять его не нужно :). Если серьёзно, сегодня я расскажу о тех изменениях, которые мы внесли в алгоритм в форке Tarantool от Picodata, и неизбежно коснусь более глубокой проблематики работы LSM-деревьев, а конкретнее – работы планировщика слияний (compaction scheduler).

https://habr.com/ru/companies/arenadata/articles/1018042/

#picodata #tarantool #vinyl #lsmдерево #lsm_tree #compaction

Claude Code's compaction discards data that's still on disk

https://github.com/anthropics/claude-code/issues/26771

#HackerNews #ClaudeCode #Compaction #Discard #Data #Disk #DataManagement #DiskUsage

----------------

🎯 AI
===================

Executive summary: OpenAI introduces three agentic primitives to make long-horizon, knowledge-work agents practical: reusable Skills (with a SKILL.md manifest), an upgraded Shell execution environment (hosted containers accessible via the Responses API), and server-side compaction for maintaining long conversational context.

Technical details:
• Skills are versioned bundles of files plus a SKILL.md manifest exposing metadata (name, description, path) that the model uses to decide invocation.
• The Shell tool provides a real terminal environment implemented as OpenAI-hosted containers (or local shells with identical semantics) that support stateful runs, tool calls, multi-turn continuations, and artifact creation.
• Compaction operates server-side to compress conversation history when context thresholds are crossed; two modes exist: automatic in-stream compaction and an explicit /responses/compact endpoint for manual control.

How it works conceptually:
• Routing: the model reads skill metadata (name, description, path) and only loads SKILL.md when it decides to invoke the skill, reducing prompt bloat.
• Execution: hosted Shell runs through the Responses API, coupling tool execution and artifact storage with the agent’s conversation state.
• Continuity: compaction preserves semantic continuity by reducing token footprint while retaining essential context for long workflows.

Best practices and patterns observed:
• Skill descriptions should be written as routing logic: concrete “Use when” vs. “Don’t use when,” explicit outputs, and success criteria.
• Include negative examples and edge cases in skill metadata to reduce misfires; Glean reported an initial ~20% drop in correct triggers that recovered after adding negative examples.
• Keep stable procedures in Skills (versioned playbooks) and leave dynamic planning in the model to avoid prompt spaghetti.

Limitations and considerations:
• Skills require careful authoring of routing metadata to avoid misrouting.
• Hosted Shell access implies controlled internet/process boundaries; security and audit controls around hosted execution environments are critical in deployment designs.

🔹 agents #skills #compaction #ResponsesAPI

🔗 Source: https://developers.openai.com/blog/skills-shell-tips/

Iceberg 1.8 + AWS Glue + Spark 3.5: как мы почти утонули в delete-файлах и что нас спасло

Рассказ о нашем опыте использования Iceberg + Spark. О том, что такое delete-файлы, конфликты транзакций и как с ними справляется операция compaction. Какие проблемы у нас возникли и как мы их решали.

https://habr.com/ru/articles/992178/

#Iceeberg #Compaction #spark

От ClickHouse к StarRocks с разделением хранения и вычислений: практический апгрейд архитектуры UBT в Trip

This is a hands-on case study of migrating Trip’s UBT from ClickHouse to StarRocks with storage–compute separation. By redesigning partitioning, enabling DataCache and MergeCommit, and backfilling history via SparkLoad, we reduced average query latency from 1.4 s to 203 ms, P95 to 800 ms, cut storage from 2.6 PB to 1.2 PB, and decreased node count from 50 to 40. We detail Compaction tuning, partitioned materialized views, and second‑level elastic scaling without data migration, and compare gohangout vs. Flink in reliability and operability. The article targets data engineers and architects running high‑load real‑time OLAP workloads.

https://habr.com/ru/articles/973230/

#StarRocks #ClickHouse #OLAP #разделение_хранения_и_вычислений #Apache_Flink #Kafka #Compaction #MergeCommit #DataCache

От ClickHouse к StarRocks с разделением хранения и вычислений: практический апгрейд архитектуры UBT в Trip

This is a hands-on case study of migrating Trip’s UBT from ClickHouse to StarRocks with storage–compute separation. By redesigning partitioning, enabling DataCache and MergeCommit, and backfilling history via SparkLoad, we reduced average query latency from 1.4 s to 203 ms, P95 to 800 ms, cut storage from 2.6 PB to 1.2 PB, and decreased node count from 50 to 40. We detail Compaction tuning, partitioned materialized views, and second‑level elastic scaling without data migration, and compare gohangout vs. Flink in reliability and operability. The article targets data engineers and architects running high‑load real‑time OLAP workloads.

https://habr.com/ru/articles/973230/

#StarRocks #ClickHouse #OLAP #разделение_хранения_и_вычислений #Apache_Flink #Kafka #Compaction #MergeCommit #DataCache

От ClickHouse к StarRocks с разделением хранения и вычислений: практический апгрейд архитектуры UBT в Trip

This is a hands-on case study of migrating Trip’s UBT from ClickHouse to StarRocks with storage–compute separation. By redesigning partitioning, enabling DataCache and MergeCommit, and backfilling history via SparkLoad, we reduced average query latency from 1.4 s to 203 ms, P95 to 800 ms, cut storage from 2.6 PB to 1.2 PB, and decreased node count from 50 to 40. We detail Compaction tuning, partitioned materialized views, and second‑level elastic scaling without data migration, and compare gohangout vs. Flink in reliability and operability. The article targets data engineers and architects running high‑load real‑time OLAP workloads.

https://habr.com/ru/articles/973230/

#StarRocks #ClickHouse #OLAP #разделение_хранения_и_вычислений #Apache_Flink #Kafka #Compaction #MergeCommit #DataCache

От ClickHouse к StarRocks с разделением хранения и вычислений: практический апгрейд архитектуры UBT в Trip

This is a hands-on case study of migrating Trip’s UBT from ClickHouse to StarRocks with storage–compute separation. By redesigning partitioning, enabling DataCache and MergeCommit, and backfilling history via SparkLoad, we reduced average query latency from 1.4 s to 203 ms, P95 to 800 ms, cut storage from 2.6 PB to 1.2 PB, and decreased node count from 50 to 40. We detail Compaction tuning, partitioned materialized views, and second‑level elastic scaling without data migration, and compare gohangout vs. Flink in reliability and operability. The article targets data engineers and architects running high‑load real‑time OLAP workloads.

https://habr.com/ru/articles/973230/

#StarRocks #ClickHouse #OLAP #разделение_хранения_и_вычислений #Apache_Flink #Kafka #Compaction #MergeCommit #DataCache

Stanford Study Shows Where – And Why – Recharge Is Uplifting Sunken Ground
--
https://sjvwater.org/stanford-study-shows-where-and-why-recharge-is-uplifting-sunken-ground/ <-- shared technical article
--
https://iopscience.iop.org/article/10.1088/1748-9326/ad5855 <-- shared paper
--
#GIS #spatial #mapping #remotesensing #groundwater #subsidence #overpumping #water #hydrology #waterresources #SanJoaquinValley #California #SanJoaquin #recharge #rechargingaquifer #aquifer #InSAR #radar #uplift #elevation #spatialanalysis #spatiotemporal #AEM #soil #geology #clay #sediment #gravel #compaction #infrastructure #canal #watersecurity #farming #agriculture

Stanford Study Shows Where – And Why – Recharge Is Uplifting Sunken Ground
--
https://sjvwater.org/stanford-study-shows-where-and-why-recharge-is-uplifting-sunken-ground/ <-- shared technical article
--
https://iopscience.iop.org/article/10.1088/1748-9326/ad5855 <-- shared paper
--
#GIS #spatial #mapping #remotesensing #groundwater #subsidence #overpumping #water #hydrology #waterresources #SanJoaquinValley #California #SanJoaquin #recharge #rechargingaquifer #aquifer #InSAR #radar #uplift #elevation #spatialanalysis #spatiotemporal #AEM #soil #geology #clay #sediment #gravel #compaction #infrastructure #canal #watersecurity #farming #agriculture

Stanford Study Shows Where – And Why – Recharge Is Uplifting Sunken Ground
--
https://sjvwater.org/stanford-study-shows-where-and-why-recharge-is-uplifting-sunken-ground/ <-- shared technical article
--
https://iopscience.iop.org/article/10.1088/1748-9326/ad5855 <-- shared paper
--
#GIS #spatial #mapping #remotesensing #groundwater #subsidence #overpumping #water #hydrology #waterresources #SanJoaquinValley #California #SanJoaquin #recharge #rechargingaquifer #aquifer #InSAR #radar #uplift #elevation #spatialanalysis #spatiotemporal #AEM #soil #geology #clay #sediment #gravel #compaction #infrastructure #canal #watersecurity #farming #agriculture

Stanford Study Shows Where – And Why – Recharge Is Uplifting Sunken Ground
--
https://sjvwater.org/stanford-study-shows-where-and-why-recharge-is-uplifting-sunken-ground/ <-- shared technical article
--
https://iopscience.iop.org/article/10.1088/1748-9326/ad5855 <-- shared paper
--
#GIS #spatial #mapping #remotesensing #groundwater #subsidence #overpumping #water #hydrology #waterresources #SanJoaquinValley #California #SanJoaquin #recharge #rechargingaquifer #aquifer #InSAR #radar #uplift #elevation #spatialanalysis #spatiotemporal #AEM #soil #geology #clay #sediment #gravel #compaction #infrastructure #canal #watersecurity #farming #agriculture

Stanford Study Shows Where – And Why – Recharge Is Uplifting Sunken Ground
--
https://sjvwater.org/stanford-study-shows-where-and-why-recharge-is-uplifting-sunken-ground/ <-- shared technical article
--
https://iopscience.iop.org/article/10.1088/1748-9326/ad5855 <-- shared paper
--
#GIS #spatial #mapping #remotesensing #groundwater #subsidence #overpumping #water #hydrology #waterresources #SanJoaquinValley #California #SanJoaquin #recharge #rechargingaquifer #aquifer #InSAR #radar #uplift #elevation #spatialanalysis #spatiotemporal #AEM #soil #geology #clay #sediment #gravel #compaction #infrastructure #canal #watersecurity #farming #agriculture

The East Coast Is Sinking [USA]
--
https://www.nytimes.com/interactive/2024/02/13/climate/flooding-sea-levels-groundwater.html <-- shared media article
--
https://doi.org/10.1093/pnasnexus/pgad426 <-- shared paper
--
#GIS #spatial #mapping #gischat #remotesensing #subsidence #EastCoast #USA #groundwater #pumping #overextraction #depletion #aquifers #waterresources #saltwaterintrusion #coast #coastal #engineeringgeology #soils #reclamation #sediments #compaction #water #hydrology #dewatering #SLR #sealevel #sealevelrise #climatechange #SLR #glacial #isostaticrebound #geology #landmovement #sedimentation #USGS
@USGS

The East Coast Is Sinking [USA]
--
https://www.nytimes.com/interactive/2024/02/13/climate/flooding-sea-levels-groundwater.html <-- shared media article
--
https://doi.org/10.1093/pnasnexus/pgad426 <-- shared paper
--
#GIS #spatial #mapping #gischat #remotesensing #subsidence #EastCoast #USA #groundwater #pumping #overextraction #depletion #aquifers #waterresources #saltwaterintrusion #coast #coastal #engineeringgeology #soils #reclamation #sediments #compaction #water #hydrology #dewatering #SLR #sealevel #sealevelrise #climatechange #SLR #glacial #isostaticrebound #geology #landmovement #sedimentation #USGS
@USGS

The East Coast Is Sinking [USA]
--
https://www.nytimes.com/interactive/2024/02/13/climate/flooding-sea-levels-groundwater.html <-- shared media article
--
https://doi.org/10.1093/pnasnexus/pgad426 <-- shared paper
--
#GIS #spatial #mapping #gischat #remotesensing #subsidence #EastCoast #USA #groundwater #pumping #overextraction #depletion #aquifers #waterresources #saltwaterintrusion #coast #coastal #engineeringgeology #soils #reclamation #sediments #compaction #water #hydrology #dewatering #SLR #sealevel #sealevelrise #climatechange #SLR #glacial #isostaticrebound #geology #landmovement #sedimentation #USGS
@USGS

The East Coast Is Sinking [USA]
--
https://www.nytimes.com/interactive/2024/02/13/climate/flooding-sea-levels-groundwater.html <-- shared media article
--
https://doi.org/10.1093/pnasnexus/pgad426 <-- shared paper
--
#GIS #spatial #mapping #gischat #remotesensing #subsidence #EastCoast #USA #groundwater #pumping #overextraction #depletion #aquifers #waterresources #saltwaterintrusion #coast #coastal #engineeringgeology #soils #reclamation #sediments #compaction #water #hydrology #dewatering #SLR #sealevel #sealevelrise #climatechange #SLR #glacial #isostaticrebound #geology #landmovement #sedimentation #USGS
@USGS

The East Coast Is Sinking [USA]
--
https://www.nytimes.com/interactive/2024/02/13/climate/flooding-sea-levels-groundwater.html <-- shared media article
--
https://doi.org/10.1093/pnasnexus/pgad426 <-- shared paper
--
#GIS #spatial #mapping #gischat #remotesensing #subsidence #EastCoast #USA #groundwater #pumping #overextraction #depletion #aquifers #waterresources #saltwaterintrusion #coast #coastal #engineeringgeology #soils #reclamation #sediments #compaction #water #hydrology #dewatering #SLR #sealevel #sealevelrise #climatechange #SLR #glacial #isostaticrebound #geology #landmovement #sedimentation #USGS
@USGS

Logging practices:
"It's basically dead" AI reveals the legacy of decades of logging in Victoria

"An AI-based analysis of 20 years of VicForests’ logging, researchers say, shows the scale of failed regeneration in Victoria’s state forests.The data, which has been shared exclusively with the ABC, suggests that 20 per cent of Victoria’s state forests have not regenerated after logging.... That’s almost 13,000 hectares of state forest the analysis found to be standing in a state of ruin."

"Now the state is shutting its native logging industry in a matter of weeks, and the forests that were once given to Victoria’s state-run logging agency, VicForests, are being returned to the public."

"By law, the logging agency is required to regenerate the areas it has logged and hand them back to the public in a healthy state. But until now, how much that has actually happened has largely been a mystery."

“VicForests is definitely doing a dodgy job, pretty much as they’ve done for the best part of the last 20 years.... We have a biodiversity problem, we have a carbon problem, we have a water problem, we have a fire risk problem,” Professor Lindenmayer
>>
https://www.abc.net.au/news/2023-12-01/ai-analysis-finds-failed-forest-regrowth-after-logging-/103153614
#logging #NativeForests #LoggingIndustry #VicForests #failure #destruction #compaction #machinery #TimberMining #regeneration #PublicOwnership #liability #StopLogging #NSWLogging #SaveTuckersNob #Bushfires #ClimateEmergency #biodiversityCrisis #extinctions

Logging practices:
"It's basically dead" AI reveals the legacy of decades of logging in Victoria

"An AI-based analysis of 20 years of VicForests’ logging, researchers say, shows the scale of failed regeneration in Victoria’s state forests.The data, which has been shared exclusively with the ABC, suggests that 20 per cent of Victoria’s state forests have not regenerated after logging.... That’s almost 13,000 hectares of state forest the analysis found to be standing in a state of ruin."

"Now the state is shutting its native logging industry in a matter of weeks, and the forests that were once given to Victoria’s state-run logging agency, VicForests, are being returned to the public."

"By law, the logging agency is required to regenerate the areas it has logged and hand them back to the public in a healthy state. But until now, how much that has actually happened has largely been a mystery."

“VicForests is definitely doing a dodgy job, pretty much as they’ve done for the best part of the last 20 years.... We have a biodiversity problem, we have a carbon problem, we have a water problem, we have a fire risk problem,” Professor Lindenmayer
>>
https://www.abc.net.au/news/2023-12-01/ai-analysis-finds-failed-forest-regrowth-after-logging-/103153614
#logging #NativeForests #LoggingIndustry #VicForests #failure #destruction #compaction #machinery #TimberMining #regeneration #PublicOwnership #liability #StopLogging #NSWLogging #SaveTuckersNob #Bushfires #ClimateEmergency #biodiversityCrisis #extinctions

Logging practices:
"It's basically dead" AI reveals the legacy of decades of logging in Victoria

"An AI-based analysis of 20 years of VicForests’ logging, researchers say, shows the scale of failed regeneration in Victoria’s state forests.The data, which has been shared exclusively with the ABC, suggests that 20 per cent of Victoria’s state forests have not regenerated after logging.... That’s almost 13,000 hectares of state forest the analysis found to be standing in a state of ruin."

"Now the state is shutting its native logging industry in a matter of weeks, and the forests that were once given to Victoria’s state-run logging agency, VicForests, are being returned to the public."

"By law, the logging agency is required to regenerate the areas it has logged and hand them back to the public in a healthy state. But until now, how much that has actually happened has largely been a mystery."

“VicForests is definitely doing a dodgy job, pretty much as they’ve done for the best part of the last 20 years.... We have a biodiversity problem, we have a carbon problem, we have a water problem, we have a fire risk problem,” Professor Lindenmayer
>>
https://www.abc.net.au/news/2023-12-01/ai-analysis-finds-failed-forest-regrowth-after-logging-/103153614
#logging #NativeForests #LoggingIndustry #VicForests #failure #destruction #compaction #machinery #TimberMining #regeneration #PublicOwnership #liability #StopLogging #NSWLogging #SaveTuckersNob #Bushfires #ClimateEmergency #biodiversityCrisis #extinctions

Logging practices:
"It's basically dead" AI reveals the legacy of decades of logging in Victoria

"An AI-based analysis of 20 years of VicForests’ logging, researchers say, shows the scale of failed regeneration in Victoria’s state forests.The data, which has been shared exclusively with the ABC, suggests that 20 per cent of Victoria’s state forests have not regenerated after logging.... That’s almost 13,000 hectares of state forest the analysis found to be standing in a state of ruin."

"Now the state is shutting its native logging industry in a matter of weeks, and the forests that were once given to Victoria’s state-run logging agency, VicForests, are being returned to the public."

"By law, the logging agency is required to regenerate the areas it has logged and hand them back to the public in a healthy state. But until now, how much that has actually happened has largely been a mystery."

“VicForests is definitely doing a dodgy job, pretty much as they’ve done for the best part of the last 20 years.... We have a biodiversity problem, we have a carbon problem, we have a water problem, we have a fire risk problem,” Professor Lindenmayer
>>
https://www.abc.net.au/news/2023-12-01/ai-analysis-finds-failed-forest-regrowth-after-logging-/103153614
#logging #NativeForests #LoggingIndustry #VicForests #failure #destruction #compaction #machinery #TimberMining #regeneration #PublicOwnership #liability #StopLogging #NSWLogging #SaveTuckersNob #Bushfires #ClimateEmergency #biodiversityCrisis #extinctions

Logging practices:
"It's basically dead" AI reveals the legacy of decades of logging in Victoria

"An AI-based analysis of 20 years of VicForests’ logging, researchers say, shows the scale of failed regeneration in Victoria’s state forests.The data, which has been shared exclusively with the ABC, suggests that 20 per cent of Victoria’s state forests have not regenerated after logging.... That’s almost 13,000 hectares of state forest the analysis found to be standing in a state of ruin."

"Now the state is shutting its native logging industry in a matter of weeks, and the forests that were once given to Victoria’s state-run logging agency, VicForests, are being returned to the public."

"By law, the logging agency is required to regenerate the areas it has logged and hand them back to the public in a healthy state. But until now, how much that has actually happened has largely been a mystery."

“VicForests is definitely doing a dodgy job, pretty much as they’ve done for the best part of the last 20 years.... We have a biodiversity problem, we have a carbon problem, we have a water problem, we have a fire risk problem,” Professor Lindenmayer
>>
https://www.abc.net.au/news/2023-12-01/ai-analysis-finds-failed-forest-regrowth-after-logging-/103153614
#logging #NativeForests #LoggingIndustry #VicForests #failure #destruction #compaction #machinery #TimberMining #regeneration #PublicOwnership #liability #StopLogging #NSWLogging #SaveTuckersNob #Bushfires #ClimateEmergency #biodiversityCrisis #extinctions