Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schedule polling for new meetings #12

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/actions/setup-poetry-env/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: 'Setup Python with Poetry'
description: 'Sets up Python environment with Poetry and installs dependencies'

inputs:
python-version:
description: 'Python version to set up'
required: false
default: '3.11'

runs:
using: "composite"
steps:
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ inputs.python-version }}

- name: Install Poetry
uses: abatilo/actions-poetry@v2

- name: Configure Poetry
run: poetry config virtualenvs.create true
shell: bash

- name: Load cached Poetry dependencies
uses: actions/cache@v3
with:
path: ~/.cache/pypoetry/virtualenvs
key: poetry-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
restore-keys: |
poetry-${{ runner.os }}-

- name: Install dependencies
run: poetry install --no-interaction --no-root
shell: bash
23 changes: 3 additions & 20 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,27 +13,10 @@ jobs:
- name: Check out repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
- name: Setup Python environment
uses: ./.github/actions/setup-poetry-env
with:
python-version: '3.11' # Adjust as needed

- name: Install Poetry
uses: abatilo/actions-poetry@v2

- name: Configure Poetry
run: poetry config virtualenvs.create true

- name: Load cached Poetry dependencies
uses: actions/cache@v3
with:
path: ~/.cache/pypoetry/virtualenvs
key: poetry-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
restore-keys: |
poetry-${{ runner.os }}-

- name: Install dependencies
run: poetry install --no-interaction --no-root
python-version: '3.11'

- name: Run tests
run: poetry run pytest
21 changes: 19 additions & 2 deletions .github/workflows/update-summary.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,29 @@
name: Update Meetings Summary

on:
schedule:
# Run every day at midnight
- cron: '0 0 * * *'
workflow_dispatch:
# Allow manual trigger

jobs:
update-summary:
runs-on: ubuntu-latest
steps:
- name: Placeholder step
run: echo "This is a placeholder workflow to allow testing on the new branch."
- name: Check out repository
uses: actions/checkout@v4

- name: Setup Python environment
uses: ./.github/actions/setup-poetry-env
with:
python-version: '3.11'

- name: Run summary script
run: poetry run python scripts/summary.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change (blocking):

Run poetry run python scripts/summary.py
  poetry run python scripts/summary.py
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.11.11/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.11/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.11/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.11/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.11/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.11/x64/lib
  
/home/runner/.cache/pypoetry/virtualenvs/tgov_scraper-85zUyotX-py3.11/bin/python: can't open file '/home/runner/work/tgov-scraper/tgov-scraper/scripts/summary.py': [Errno 2] No such file or directory
Error: Process completed with exit code 2.

Comment on lines +22 to +23
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change (blocking): llama_summary.py at least needs GITHUB_TOKEN available. I think you have to explicitly pass the token from the action environment to the action.

Suggested change
- name: Run summary script
run: poetry run python scripts/summary.py
- name: Run summary script
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: poetry run python scripts/summary.py


- name: Commit and push changes
uses: stefanzweifel/git-auto-commit-action@v4
with:
commit_message: "Update meetings summary data"
file_pattern: "data/meetings.jsonl data/summaries.jsonl website/data/meetings.jsonl website/data/summaries.jsonl"
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ data/audio/
data/video/
data/transcripts/
models/
# Include specific directories
!src/models/

notebooks/.ipynb_checkpoints/

# Python
Expand Down
2,384 changes: 2,384 additions & 0 deletions data/meetings.jsonl
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change (blocking): don't check the *.jsonl file into the repo or we'll have to git commit them into the repo to update them. They should be put into the storage destination like S3 or something.

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions data/summaries.jsonl
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change (blocking): don't check the *.jsonl file into the repo or we'll have to git commit them into the repo to update them. They should be put into the storage destination like S3 or something.

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{"meeting": {"meeting": "Regular Council Meeting", "date": "2025-03-12T17:00:00-05:00", "date_display": "March 12, 2025 - 5:00 PM", "duration": "01h\u00a038m", "agenda": "https://tulsa-ok.granicus.com/AgendaViewer.php?view_id=4&clip_id=6482", "video": "https://tulsa-ok.granicus.com/MediaPlayer.php?view_id=4&clip_id=6482"}, "summary": "**Summary of Tulsa City Council Meeting - March 12, 2025**\n\nThe Tulsa City Council held a regular meeting on March 12, 2025, to discuss and vote on various agenda items. The key topics discussed included:\n\n1. **Appointments and Reappointments**: The council considered appointments to the Board of Adjustment and the Sales Tax Overview Committee.\n2. **Public Hearings**: A public hearing was held to discuss the proposed Amended and Restated Tulsa International Airport Economic Project Plan, which aims to revise increment district boundaries and update project costs.\n3. **Mayor's Items**: The mayor reported on community events and city activities, and the council considered several resolutions, including payments to plaintiffs and attorneys in various cases.\n4. **Special Event Applications**: The council reviewed and considered approval of several special event applications, including the McNellie's St. Patrick's Day Celebration, Kilkenny's Irish Pub Block Party, and the Tulsa Juneteenth 5K and 1 Mile Run.\n5. **Ordinances**: The council considered several ordinances, including those related to utility easement closures, public way closures, and revisions to the city's code of ordinances.\n\n**Notable Public Comments**:\n\n* Bobbi Drew expressed concerns regarding the installation and placement of sidewalks on E. Newton St., citing lack of notification and lack of speed humps.\n\n**Decisions Made**:\n\n* The council approved the appointment of Rosalind Williams to the Board of Adjustment and the reappointment of Linda Jenkins to the Sales Tax Overview Committee.\n* The council considered and voted on several resolutions, including payments to plaintiffs and attorneys in various cases.\n* The council approved several special event applications, including the McNellie's St. Patrick's Day Celebration and the Kilkenny's Irish Pub Block Party.\n\nThe meeting concluded with a hearing of appeals and public comments, and the council adjourned until the next scheduled meeting.", "summarization_date": "2025-03-19T18:01:35.234584+00:00", "needs_summarization": false}
{"meeting": {"meeting": "Council Public Works Committee", "date": "2025-03-12T14:32:00-05:00", "date_display": "March 12, 2025 - 2:32 PM", "duration": "01h\u00a013m", "agenda": "https://tulsa-ok.granicus.com/AgendaViewer.php?view_id=4&clip_id=6481", "video": "https://tulsa-ok.granicus.com/MediaPlayer.php?view_id=4&clip_id=6481"}, "summary": "**Summary of Tulsa City Council Public Works Committee Meeting**\n\nThe Tulsa City Council Public Works Committee meeting was held on March 12, 2025, at 2:30 PM. The meeting covered various topics related to public works, including reappointments to city commissions, easement closures, and license agreements.\n\n**Key Topics Discussed:**\n\n1. Reappointments: The committee considered reappointing Jameelah Stuckey to the Greater Tulsa Area African-American Affairs Commission and Steve King to the Sales Tax Overview Committee.\n2. Easement Closures: The committee reviewed several ordinances related to closing utility easements and public ways for future development and expansion of the VA Hospital.\n3. License Agreements: The committee discussed license agreements between the City of Tulsa and property owners for installations within the street right-of-way.\n4. Abandoned Facilities: Representatives from the Public Works Department discussed regulations and operating considerations for abandoned or unused facilities in the rights of way.\n5. Hazmat Billing Practices: The committee reviewed billing practices for hazmat incidents with representatives from the Tulsa Fire Department.\n\n**Decisions Made:**\n\nWhile the exact decisions are not specified, the committee likely voted on the reappointments, easement closures, and license agreements. The discussions with representatives from the Public Works Department and Tulsa Fire Department may have led to further review or action on the topics of abandoned facilities and hazmat billing practices.\n\n**Notable Public Comments:**\n\nNo notable public comments are mentioned in the agenda. However, the meeting was open to the public, and citizens may have provided input on the discussed topics.\n\nOverall, the meeting focused on routine public works business, including reappointments, easement closures, and license agreements. The discussions with city departments may have laid the groundwork for future actions or policy changes related to abandoned facilities and hazmat billing practices.", "summarization_date": "2025-03-19T18:01:45.256267+00:00", "needs_summarization": false}
{"meeting": {"meeting": "Hispanic/LatinX Affairs Commission", "date": "2025-03-13T18:01:00-05:00", "date_display": "March 13, 2025 - 6:01 PM", "duration": "01h\u00a010m", "agenda": null, "video": "https://tulsa-ok.granicus.com/MediaPlayer.php?view_id=4&clip_id=6484"}, "summary": "**Government Meeting Summary**\n\nDue to the unavailability of the meeting agenda, a general summary of typical government meeting proceedings is provided. \n\nIn a standard government meeting, key topics often include updates on local infrastructure projects, discussions on budget allocations, and reviews of proposed policies or ordinances. These meetings typically involve presentations by government officials, public works departments, or external experts, followed by discussions and debates among council members or committee representatives.\n\nDecisions made during these meetings can range from approving or rejecting proposals, allocating funds for specific projects, or establishing new regulations. The meetings often provide a platform for council members to raise concerns, address community issues, and engage in dialogue with fellow officials.\n\nPublic comments are a crucial aspect of these meetings, allowing citizens to express their opinions, concerns, or support for specific issues. These comments can influence the decision-making process and help ensure that the government is responsive to the needs and preferences of the community.\n\nIn general, government meetings aim to promote transparency, foster collaboration among stakeholders, and drive progress on initiatives that benefit the community. Although the specifics of this meeting are unknown, it is likely that the discussion centered around issues of local importance, with a focus on making informed decisions that serve the public interest.", "summarization_date": "2025-03-19T18:01:52.399434+00:00", "needs_summarization": false}
{"meeting": {"meeting": "HUD Community Development Committee", "date": "2025-03-13T17:02:00-05:00", "date_display": "March 13, 2025 - 5:02 PM", "duration": "00h\u00a043m", "agenda": null, "video": "https://tulsa-ok.granicus.com/MediaPlayer.php?view_id=4&clip_id=6483"}, "summary": "**Government Meeting Summary**\n\nIn the absence of a specific agenda, this summary will provide a general overview of typical government meeting discussions and outcomes.\n\nA government meeting was held, where local officials and stakeholders gathered to discuss various community matters. Although the exact agenda is unavailable, these meetings usually cover topics such as:\n\n1. **Infrastructure and Development**: Discussions on ongoing or proposed infrastructure projects, zoning regulations, and land use planning.\n2. **Budget and Finance**: Review of budget allocations, financial reports, and funding requests for community programs and services.\n3. **Public Services and Safety**: Updates on public health initiatives, law enforcement, and emergency response services.\n4. **Environmental and Community Concerns**: Addressing concerns related to environmental sustainability, waste management, and community engagement initiatives.\n\nDuring the meeting, officials likely made decisions on:\n\n* Approving or rejecting proposals for new developments or infrastructure projects\n* Allocating funds for community programs and services\n* Implementing new policies or regulations to address community concerns\n\nThe meeting may have also included a public comment period, where citizens had the opportunity to express their opinions, concerns, or suggestions on various topics. Notable public comments may have focused on issues such as:\n\n* Concerns about local traffic and transportation\n* Support or opposition to proposed developments\n* Requests for improved community services or amenities\n\nAs the meeting concluded, officials may have provided updates on the status of ongoing projects, outlined next steps for proposed initiatives, and established timelines for implementing new policies or regulations. The meeting's outcome will likely influence future community development, service delivery, and policy decisions.", "summarization_date": "2025-03-19T18:02:01.804586+00:00", "needs_summarization": false}
{"meeting": {"meeting": "Human Rights Commission", "date": "2025-03-17T15:31:00-05:00", "date_display": "March 17, 2025 - 3:31 PM", "duration": "00h\u00a044m", "agenda": null, "video": "https://tulsa-ok.granicus.com/MediaPlayer.php?view_id=4&clip_id=6490"}, "summary": "**Government Meeting Summary**\n\nDue to the unavailability of the meeting agenda, a general summary of typical government meeting proceedings is provided below.\n\nThe government meeting likely commenced with the introduction of attendees, followed by the review and approval of previous meeting minutes. The main agenda items typically include discussions on ongoing projects, policy updates, and budget allocations.\n\nKey topics often discussed in government meetings may involve:\n\n1. Infrastructure development and public works projects\n2. Budget reviews and financial planning\n3. Public health and safety initiatives\n4. Environmental conservation and sustainability efforts\n5. Community development and social services\n\nDecisions made during the meeting may have included:\n\n1. Approval of project proposals and funding allocations\n2. Adoption of new policies or amendments to existing ones\n3. Appointment of committee members or task forces\n4. Review and approval of contractual agreements\n\nNotable public comments often focus on:\n\n1. Concerns about local issues, such as traffic, noise pollution, or public safety\n2. Support or opposition to proposed projects or policies\n3. Requests for improved community services or facilities\n4. Suggestions for enhancing transparency and public engagement in government decision-making processes\n\nIn the absence of specific details, it is assumed that the meeting followed a standard format, with officials and attendees engaging in constructive discussions to address community needs and make informed decisions. A detailed summary would be possible with access to the meeting agenda and minutes.", "summarization_date": "2025-03-19T18:02:09.389871+00:00", "needs_summarization": false}
{"meeting": {"meeting": "Indian Affairs Commission", "date": "2025-03-18T18:03:00-05:00", "date_display": "March 18, 2025 - 6:03 PM", "duration": "00h\u00a042m", "agenda": null, "video": "https://tulsa-ok.granicus.com/MediaPlayer.php?view_id=4&clip_id=6492"}, "summary": "**Government Meeting Summary**\n\nDue to the unavailability of the agenda, a general summary of a typical government meeting is provided. \n\nIn a standard government meeting, key topics often include discussions on local policies, budget allocations, infrastructure development, and community concerns. These meetings usually involve council members, government officials, and sometimes invited experts who present information and insights on various issues.\n\nDecisions made during these meetings can range from approving new projects, resolving community issues, and adopting policies that impact the local community. The meeting may also include a public comment period, where citizens can express their opinions, concerns, or suggestions on the topics being discussed.\n\nSome common discussions in government meetings include:\n\n1. Budget reviews and allocations for various public services and projects.\n2. Updates on ongoing infrastructure projects, such as road construction, public transportation, or community facilities.\n3. Policy discussions on topics like public safety, environmental conservation, and economic development.\n4. Presentations from experts on specific issues, such as healthcare, education, or social services.\n\nPublic comments often provide valuable insights into community concerns and opinions on the topics being discussed. These comments can influence the decisions made by government officials and help shape the direction of local policies.\n\nWithout the specific agenda, it is challenging to provide a more detailed summary. However, this general overview should give an idea of the typical discussions and decision-making processes that occur in a government meeting.", "summarization_date": "2025-03-19T18:02:16.423813+00:00", "needs_summarization": false}
Loading
Loading