home.social

#phonecall — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #phonecall, aggregated by home.social.

  1. Building a Local AI Information-Gathering Phone Line

    https://youtu.be/Yodq7gG_F3I

    Modern voice agents are often presented as cloud services: a caller speaks, a remote platform transcribes the audio, a hosted language model generates a response, and another service synthesizes the reply.

    For this project, I wanted to explore a different approach.

    The goal was to build a locally controlled conversational phone-line prototype capable of gathering structured information from a caller while maintaining context throughout the interaction.

    The system uses a locally hosted Gemma 4 model for dialogue, local speech recognition for transcription, and Kokoro 82M for text-to-speech. A custom Python application coordinates the entire interaction and presents it through a phone-style interface.

    The result is an automated intake agent named Eva.

    The Test Scenario

    For the demonstration, Eva was configured to behave like a corporate information-gathering agent.

    The caller began with a simple greeting. Eva then guided the conversation through a sequence of intake questions:

    • Full name
    • Company or organization
    • Callback number
    • Email address
    • Case, ticket, order, or reference number
    • General reason for the call
    • Product or service involved
    • Date and approximate time of the reported issue

    The caller did not have to complete a rigid form or choose numbered menu options. Instead, the information was collected through natural conversation.

    A portion of the interaction looked like this:

    Eva: Hello. Thank you for contacting us. My name is Eva. To begin, may I please have your full name?
    Caller: John Michael Smith.
    Eva: Thank you, John Michael Smith. May I also have the company or organization you are calling from?

    As the conversation continued, Eva retained the previously supplied information and advanced to the next relevant question.

    When the caller explained that they were upset about new automated systems, Eva did not restart the intake process or lose track of the conversation. She asked a follow-up question to determine which product or service was involved.

    That contextual continuity is the most important part of the demonstration.

    The Core Architecture

    The system is built as a real-time conversational pipeline:

     Microphone input        ↓  Speech-to-text        ↓  Conversation controller        ↓  Gemma 4 through LM Studio        ↓  Kokoro 82M text-to-speech        ↓  Audio playback  

    Each component has a distinct responsibility.

    The speech-to-text layer converts the caller’s voice into text. The conversation controller maintains the active call state and sends the transcript to the language model. Gemma 4 evaluates the current message alongside the prior conversation and produces the next response. Kokoro 82M then converts that response into spoken audio.

    The Python application ties these systems together.

    It manages:

    • Microphone capture
    • Voice activity detection
    • Turn-taking
    • Conversation history
    • System prompts
    • Model requests
    • Voice generation
    • Audio playback
    • Call state
    • Session termination
    • Persistent memory
    • Transcript logging

    The language model is only one component of the system. The surrounding orchestration is what makes the experience feel like a coherent call rather than a sequence of unrelated AI requests.

    Why Conversation History Matters

    A language model does not automatically remember everything that happened earlier in a call.

    The application must preserve the conversation and provide the relevant history with each new request.

    Without that context, an information-gathering agent might repeatedly ask for the caller’s name, forget which fields were already completed, or ask questions in an inconsistent order.

    In this prototype, the conversation controller tracks both sides of the exchange:

     System instructions  Caller message  Agent response  Caller message  Agent response  

    That history allows the model to understand which information has already been collected and what still needs to be asked.

    It also enables more natural follow-up behavior.

    For example, when the caller says:

    “I am upset about the new automated systems.”

    Eva can interpret that statement as the reason for the call and ask which product is affected, rather than simply continuing through an unrelated checklist.

    Structured Behavior Without a Traditional Form

    One of the interesting aspects of this design is that the model can follow a structured intake objective without forcing the caller through a conventional form.

    The system prompt defines the agent’s responsibilities, tone, required information, and conversational boundaries.

    The agent can then gather the same fields a form would collect while allowing the caller to speak naturally.

    This creates a hybrid between two familiar systems:

     Rigid automated phone menu              +  Human-style conversational intake  

    The structured requirements remain in place, but the interface becomes conversational.

    That can be useful in situations where callers may not know how their problem should be categorized before they begin speaking.

    Local Inference and Control

    The language model is served locally through LM Studio rather than being accessed through a commercial cloud API.

    This provides direct control over:

    • Model selection
    • System prompts
    • Conversation history
    • Context length
    • Generation settings
    • Data retention
    • Personality configuration
    • Memory behavior

    The text-to-speech component is also hosted locally.

    That means the central conversation does not depend on sending every transcript and model response to an external AI provider.

    Local inference introduces its own technical requirements, including GPU resources, model management, latency optimization, and service coordination. However, it also gives the developer considerably more control over the complete conversational stack.

    Voice Activity Detection

    The caller does not need to press a push-to-talk button during normal operation.

    The application monitors the microphone and determines when speech begins and ends.

    This process is known as voice activity detection.

    The system moves through several internal states:

     Listening     ↓  Speech detected     ↓  Recording     ↓  Silence detected     ↓  Transcribing     ↓  Thinking     ↓  Speaking     ↓  Listening  

    This state-based design is important because the microphone, speech model, language model, and voice model all operate at different speeds.

    The application must know which component currently owns the interaction.

    It must also prevent the microphone from transcribing the AI’s own voice as new caller input.

    Persistent Memory

    The application also includes persistent conversational memory.

    This allows the agent to retain selected context between separate sessions rather than beginning every call as a completely blank system.

    The memory architecture separates several types of information:

    • Current-call conversation
    • Recent message history
    • Summaries of earlier calls
    • Long-term memories
    • Archived transcripts

    This distinction matters because sending every previous conversation back into the model would eventually become inefficient.

    Instead, recent dialogue can remain verbatim while older interactions are summarized. Important details can be stored separately and included only when relevant.

    The application, not the language model, owns this memory.

    That is an important architectural principle.

    The model generates language, but the surrounding software determines what is saved, retrieved, discarded, or presented as context.

    The Interface

    The application was designed to resemble a live phone call rather than a conventional chatbot window.

    The interface includes:

    • Start and hang-up controls
    • Call timer
    • Current system state
    • Editable system prompt
    • Agent profiles
    • Live transcription
    • Model responses
    • Diagnostic timing information
    • Persistent memory controls
    • A reactive circular audio visualization

    The visualizer changes according to the active state.

    The caller’s speech is represented in yellow. Model processing uses blue and purple states. The generated AI voice is shown in green.

    This is primarily an aesthetic feature, but it also provides immediate feedback about what the system is doing.

    A user can see whether the application is listening, transcribing, waiting for the model, generating speech, or playing audio.

    What This Demonstration Proves

    This experiment demonstrates that a locally hosted language model can operate as the conversational core of an automated information-gathering line.

    More specifically, it shows that the system can:

    • Listen for natural speech
    • Convert speech into text
    • Preserve conversational context
    • Collect information in sequence
    • Ask relevant follow-up questions
    • Produce spoken responses
    • Maintain a consistent agent identity
    • Store and recall previous conversational information
    • Operate through a custom local interface

    The prototype is not intended to replace a production call center in its current form.

    A production deployment would require additional work in areas such as authentication, encryption, regulatory compliance, consent, auditing, data validation, failure recovery, telephony integration, and human escalation.

    But the central conversational mechanism is functional.

    The Larger Possibility

    The broader significance of this project is not limited to corporate intake.

    The same architecture could support many types of locally controlled voice agents:

    • Technical support intake
    • Appointment pre-screening
    • Internal help desks
    • Interactive training systems
    • Voice-based surveys
    • Game characters
    • Accessibility tools
    • Personal assistants
    • Research interviews
    • Educational tutors

    The underlying pattern remains the same:

     Listen  Understand  Maintain context  Respond  Remember  

    What changes is the system prompt, the information being collected, the voice, and the surrounding workflow.

    Final Thoughts

    The project began as a simple idea: speak into a microphone, send the transcript to a local model, and play the response through a local voice engine.

    Once persistent memory, automatic voice detection, agent profiles, structured prompts, call archives, and state management were added, it became something more substantial.

    It became a locally controlled conversational-agent platform.

    The most important lesson from the project is that the model itself is not the complete system.

    A useful voice agent emerges from the coordination of perception, context, reasoning, memory, expression, and interface design.

    Gemma 4 provides the conversational intelligence.

    Kokoro 82M provides the voice.

    The Python application provides the structure that allows them to behave like a single coherent agent.

    -Me 7-15-2026

    #agent #ai #artificialIntelligence #assistant #call #llm #operator #phone #phonecall #technology #telephone
  2. Building a Local AI Information-Gathering Phone Line

    https://youtu.be/Yodq7gG_F3I

    Modern voice agents are often presented as cloud services: a caller speaks, a remote platform transcribes the audio, a hosted language model generates a response, and another service synthesizes the reply.

    For this project, I wanted to explore a different approach.

    The goal was to build a locally controlled conversational phone-line prototype capable of gathering structured information from a caller while maintaining context throughout the interaction.

    The system uses a locally hosted Gemma 4 model for dialogue, local speech recognition for transcription, and Kokoro 82M for text-to-speech. A custom Python application coordinates the entire interaction and presents it through a phone-style interface.

    The result is an automated intake agent named Eva.

    The Test Scenario

    For the demonstration, Eva was configured to behave like a corporate information-gathering agent.

    The caller began with a simple greeting. Eva then guided the conversation through a sequence of intake questions:

    • Full name
    • Company or organization
    • Callback number
    • Email address
    • Case, ticket, order, or reference number
    • General reason for the call
    • Product or service involved
    • Date and approximate time of the reported issue

    The caller did not have to complete a rigid form or choose numbered menu options. Instead, the information was collected through natural conversation.

    A portion of the interaction looked like this:

    Eva: Hello. Thank you for contacting us. My name is Eva. To begin, may I please have your full name?
    Caller: John Michael Smith.
    Eva: Thank you, John Michael Smith. May I also have the company or organization you are calling from?

    As the conversation continued, Eva retained the previously supplied information and advanced to the next relevant question.

    When the caller explained that they were upset about new automated systems, Eva did not restart the intake process or lose track of the conversation. She asked a follow-up question to determine which product or service was involved.

    That contextual continuity is the most important part of the demonstration.

    The Core Architecture

    The system is built as a real-time conversational pipeline:

     Microphone input        ↓  Speech-to-text        ↓  Conversation controller        ↓  Gemma 4 through LM Studio        ↓  Kokoro 82M text-to-speech        ↓  Audio playback  

    Each component has a distinct responsibility.

    The speech-to-text layer converts the caller’s voice into text. The conversation controller maintains the active call state and sends the transcript to the language model. Gemma 4 evaluates the current message alongside the prior conversation and produces the next response. Kokoro 82M then converts that response into spoken audio.

    The Python application ties these systems together.

    It manages:

    • Microphone capture
    • Voice activity detection
    • Turn-taking
    • Conversation history
    • System prompts
    • Model requests
    • Voice generation
    • Audio playback
    • Call state
    • Session termination
    • Persistent memory
    • Transcript logging

    The language model is only one component of the system. The surrounding orchestration is what makes the experience feel like a coherent call rather than a sequence of unrelated AI requests.

    Why Conversation History Matters

    A language model does not automatically remember everything that happened earlier in a call.

    The application must preserve the conversation and provide the relevant history with each new request.

    Without that context, an information-gathering agent might repeatedly ask for the caller’s name, forget which fields were already completed, or ask questions in an inconsistent order.

    In this prototype, the conversation controller tracks both sides of the exchange:

     System instructions  Caller message  Agent response  Caller message  Agent response  

    That history allows the model to understand which information has already been collected and what still needs to be asked.

    It also enables more natural follow-up behavior.

    For example, when the caller says:

    “I am upset about the new automated systems.”

    Eva can interpret that statement as the reason for the call and ask which product is affected, rather than simply continuing through an unrelated checklist.

    Structured Behavior Without a Traditional Form

    One of the interesting aspects of this design is that the model can follow a structured intake objective without forcing the caller through a conventional form.

    The system prompt defines the agent’s responsibilities, tone, required information, and conversational boundaries.

    The agent can then gather the same fields a form would collect while allowing the caller to speak naturally.

    This creates a hybrid between two familiar systems:

     Rigid automated phone menu              +  Human-style conversational intake  

    The structured requirements remain in place, but the interface becomes conversational.

    That can be useful in situations where callers may not know how their problem should be categorized before they begin speaking.

    Local Inference and Control

    The language model is served locally through LM Studio rather than being accessed through a commercial cloud API.

    This provides direct control over:

    • Model selection
    • System prompts
    • Conversation history
    • Context length
    • Generation settings
    • Data retention
    • Personality configuration
    • Memory behavior

    The text-to-speech component is also hosted locally.

    That means the central conversation does not depend on sending every transcript and model response to an external AI provider.

    Local inference introduces its own technical requirements, including GPU resources, model management, latency optimization, and service coordination. However, it also gives the developer considerably more control over the complete conversational stack.

    Voice Activity Detection

    The caller does not need to press a push-to-talk button during normal operation.

    The application monitors the microphone and determines when speech begins and ends.

    This process is known as voice activity detection.

    The system moves through several internal states:

     Listening     ↓  Speech detected     ↓  Recording     ↓  Silence detected     ↓  Transcribing     ↓  Thinking     ↓  Speaking     ↓  Listening  

    This state-based design is important because the microphone, speech model, language model, and voice model all operate at different speeds.

    The application must know which component currently owns the interaction.

    It must also prevent the microphone from transcribing the AI’s own voice as new caller input.

    Persistent Memory

    The application also includes persistent conversational memory.

    This allows the agent to retain selected context between separate sessions rather than beginning every call as a completely blank system.

    The memory architecture separates several types of information:

    • Current-call conversation
    • Recent message history
    • Summaries of earlier calls
    • Long-term memories
    • Archived transcripts

    This distinction matters because sending every previous conversation back into the model would eventually become inefficient.

    Instead, recent dialogue can remain verbatim while older interactions are summarized. Important details can be stored separately and included only when relevant.

    The application, not the language model, owns this memory.

    That is an important architectural principle.

    The model generates language, but the surrounding software determines what is saved, retrieved, discarded, or presented as context.

    The Interface

    The application was designed to resemble a live phone call rather than a conventional chatbot window.

    The interface includes:

    • Start and hang-up controls
    • Call timer
    • Current system state
    • Editable system prompt
    • Agent profiles
    • Live transcription
    • Model responses
    • Diagnostic timing information
    • Persistent memory controls
    • A reactive circular audio visualization

    The visualizer changes according to the active state.

    The caller’s speech is represented in yellow. Model processing uses blue and purple states. The generated AI voice is shown in green.

    This is primarily an aesthetic feature, but it also provides immediate feedback about what the system is doing.

    A user can see whether the application is listening, transcribing, waiting for the model, generating speech, or playing audio.

    What This Demonstration Proves

    This experiment demonstrates that a locally hosted language model can operate as the conversational core of an automated information-gathering line.

    More specifically, it shows that the system can:

    • Listen for natural speech
    • Convert speech into text
    • Preserve conversational context
    • Collect information in sequence
    • Ask relevant follow-up questions
    • Produce spoken responses
    • Maintain a consistent agent identity
    • Store and recall previous conversational information
    • Operate through a custom local interface

    The prototype is not intended to replace a production call center in its current form.

    A production deployment would require additional work in areas such as authentication, encryption, regulatory compliance, consent, auditing, data validation, failure recovery, telephony integration, and human escalation.

    But the central conversational mechanism is functional.

    The Larger Possibility

    The broader significance of this project is not limited to corporate intake.

    The same architecture could support many types of locally controlled voice agents:

    • Technical support intake
    • Appointment pre-screening
    • Internal help desks
    • Interactive training systems
    • Voice-based surveys
    • Game characters
    • Accessibility tools
    • Personal assistants
    • Research interviews
    • Educational tutors

    The underlying pattern remains the same:

     Listen  Understand  Maintain context  Respond  Remember  

    What changes is the system prompt, the information being collected, the voice, and the surrounding workflow.

    Final Thoughts

    The project began as a simple idea: speak into a microphone, send the transcript to a local model, and play the response through a local voice engine.

    Once persistent memory, automatic voice detection, agent profiles, structured prompts, call archives, and state management were added, it became something more substantial.

    It became a locally controlled conversational-agent platform.

    The most important lesson from the project is that the model itself is not the complete system.

    A useful voice agent emerges from the coordination of perception, context, reasoning, memory, expression, and interface design.

    Gemma 4 provides the conversational intelligence.

    Kokoro 82M provides the voice.

    The Python application provides the structure that allows them to behave like a single coherent agent.

    -Me 7-15-2026

    #ai #artificialIntelligence #technology #phone #agent #llm #operator #telephone #call #phonecall #assistant
  3. Gee how much annoying "call on hold" muzak is .... why can't we just have the ordinary beeps like in the good old days.

    This time it is a 1 minute and 40 sec snippet going on in a loop of some distorted piano-ish jazz. And the phone queue has given me this torture 8 times already ... and still running ....

    *sigh*

    PICK UP THE PHONE NOW!

    #phonecall #dont_kill_me #ux

  4. Gee how much annoying "call on hold" muzak is .... why can't we just have the ordinary beeps like in the good old days.

    This time it is a 1 minute and 40 sec snippet going on in a loop of some distorted piano-ish jazz. And the phone queue has given me this torture 8 times already ... and still running ....

    *sigh*

    PICK UP THE PHONE NOW!

    #phonecall #dont_kill_me #ux

  5. Gee how much annoying "call on hold" muzak is .... why can't we just have the ordinary beeps like in the good old days.

    This time it is a 1 minute and 40 sec snippet going on in a loop of some distorted piano-ish jazz. And the phone queue has given me this torture 8 times already ... and still running ....

    *sigh*

    PICK UP THE PHONE NOW!

    #phonecall #dont_kill_me #ux

  6. Gee how much annoying "call on hold" muzak is .... why can't we just have the ordinary beeps like in the good old days.

    This time it is a 1 minute and 40 sec snippet going on in a loop of some distorted piano-ish jazz. And the phone queue has given me this torture 8 times already ... and still running ....

    *sigh*

    PICK UP THE PHONE NOW!

    #phonecall #dont_kill_me #ux

  7. Gee how much annoying "call on hold" muzak is .... why can't we just have the ordinary beeps like in the good old days.

    This time it is a 1 minute and 40 sec snippet going on in a loop of some distorted piano-ish jazz. And the phone queue has given me this torture 8 times already ... and still running ....

    *sigh*

    PICK UP THE PHONE NOW!

    #phonecall #dont_kill_me #ux

  8. It's amazing the way that #God puts people on our path. This #officer told Jackson to seek out God, no matter the reason. So #thankful he made that #phonecall. #testimony

  9. Today is a terrible brain-day but I managed to finish a comic this week, and posted it just now to Tapastic - and even though it is about IH, I think my neurodivergent peeps would also like this one. (:

    Comic is FREE to read and sub to
    ALWAYS in Tapas:
    tapas.io/episode/3900320

    And in my Ko-fi:
    ko-fi.com/i/IP4T120ELUZ

    #comic #art #disability #neurodivergent #neurodivergence #phonecall #disabilitycomic #disabledartist #idiopathichypersomnia #sleepdisorder #brainfog

  10. Today is a terrible brain-day but I managed to finish a comic this week, and posted it just now to Tapastic - and even though it is about IH, I think my neurodivergent peeps would also like this one. (:

    Comic is FREE to read and sub to
    ALWAYS in Tapas:
    tapas.io/episode/3900320

    And in my Ko-fi:
    ko-fi.com/i/IP4T120ELUZ

    #comic #art #disability #neurodivergent #neurodivergence #phonecall #disabilitycomic #disabledartist #idiopathichypersomnia #sleepdisorder #brainfog

  11. Just off a #PhoneCall about an important matter. (Funding.)

    The quality of the audio was so poor.
    Syllables and words often smearing.

    Seems to happen quite frequently, but far from "all the time".

    I'm thinking the algorithims they're using to shrink the bandwidth of the audio signals are not suitable for the environment they're being used in.

    #Enshitification
    #PhoneCallAudioQuality
    #PhoneAudio
    #AudioQuality
    #ConsumingIn2025

  12. Just off a #PhoneCall about an important matter. (Funding.)

    The quality of the audio was so poor.
    Syllables and words often smearing.

    Seems to happen quite frequently, but far from "all the time".

    I'm thinking the algorithims they're using to shrink the bandwidth of the audio signals are not suitable for the environment they're being used in.

    #Enshitification
    #PhoneCallAudioQuality
    #PhoneAudio
    #AudioQuality
    #ConsumingIn2025