Commit Graph

206 Commits

Author SHA1 Message Date
Swifty
2de5e3dd83
feat(platform): Agent Store V2 (#8874)
# 🌎 Overview

AutoGPT Store Version 2 expands on the Pre-Store by enhancing agent
discovery, providing richer content presentation, and introducing new
user engagement features. The focus is on creating a visually appealing
and interactive marketplace that allows users to explore and evaluate
agents through images, videos, and detailed descriptions.

### Vision

To create a visually compelling and interactive open-source marketplace
for autonomous AI agents, where users can easily discover, evaluate, and
interact with agents through media-rich listings, ratings, and version
history.

### Objectives

📊 Incorporate visuals (icons, images, videos) into agent listings.
 Introduce a rating system and agent run count.
🔄 Provide version history and update logs from creators.
🔍 Improve user experience with advanced search and filtering features.

### Changes 🏗️

<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [ ] ...

<details>
  <summary>Example test plan</summary>
  
  - [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
  - [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
  - [ ] Edit an agent from monitor, and confirm it executes correctly
</details>

#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)

<details>
  <summary>Examples of configuration changes</summary>

  - Changing ports
  - Adding new services that need to communicate with each other
  - Secrets or environment variable changes
  - New or infrastructure changes such as databases
</details>

---------

Co-authored-by: Bently <tomnoon9@gmail.com>
Co-authored-by: Aarushi <aarushik93@gmail.com>
2024-12-13 16:35:02 +00:00
Reinier van der Leer
582e12c766
ci(frontend): Speed up test jobs (#8949)
- Resolves #8948

### Changes 🏗️

- Parallelize frontend test job into a per-browser matrix
- Speed up "Free Disk Space" step by disabling removal of large system
packages
2024-12-12 19:54:05 +00:00
Reinier van der Leer
0272d87af3
ci(backend): Add poetry.lock check (#8885)
- Resolves #8884

We need to prevent breaking updates to dependency version requirements
of `autogpt_libs`.
`autogpt_libs/pytroject.toml` and `backend/poetry.lock` are loosely
coupled, and to ensure they stay in sync we need an extra check.

For now I'm also reverting the breaking update of #8787, otherwise this
added CI check will immediately fail.

### Changes
- ci(backend): Add `poetry.lock` check
- Revert "build(deps): bump pydantic from 2.9.2 to 2.10.2 in
/autogpt_platform/autogpt_libs in the production-dependencies group
across 1 directory (#8787)"
2024-12-05 18:41:59 +00:00
Reinier van der Leer
5ccfb8e4c6
dx(backend): Fix linting & formatting for autogpt_libs (#8860)
- Resolves #8859
- Follow-up to #8751

### Changes
- Add `autogpt_libs` to the backend CI path filter
- Add `ruff format` step for `autogpt_libs` to `linter.py` and
`pre-commit` config
- Run `poetry run format` with the new setup
2024-12-03 11:34:07 +00:00
Reinier van der Leer
30bb9a3d72
dx: Fix dependabot PR/commit titles (#8841)
Dependabot's commit messages are bulky and don't use our commit message
scopes. Although not fully customizable, this partially fixes it.

### Changes 🏗️

- Fix dependabot commit message scopes

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2024-12-02 10:39:10 +00:00
Reinier van der Leer
ab3643388f
ci: Fix workflow status checker script to work with merge_group runs 2024-11-27 19:25:36 +01:00
Reinier van der Leer
845c8c51e5
ci: Add merge_group trigger to status checker 2024-11-27 19:15:28 +01:00
Reinier van der Leer
118fdeeb1d
ci: Add merge_group triggers 2024-11-27 17:57:44 +01:00
dependabot[bot]
772baff6db
build(deps): bump docker/build-push-action from 2 to 6 (#8465)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 2 to 6.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v2...v6)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2024-11-27 22:52:58 +07:00
Nicholas Tindle
92bfbfad57
feat: generate simple auth tests (#8709) 2024-11-20 15:21:16 +00:00
Aarushi
520d0ca0e4
delete infra folder (#8555)
* delete infra folder

* remove ci
2024-11-19 14:21:57 +00:00
Nicholas Tindle
ea214d9168
ci: fix classic ci (#8338)
* ci(frontend,backend,classic): update branch from develop to dev

* ci(frontend, infra): enable ci on other tools

* Update classic-autogpt-docker-ci.yml

* fix: don't error if the folder exists

* fix: drop bad test

* Revert "fix: drop bad test"

This reverts commit c478d3cf4c.

* fix: turn off the correct test 👀

* fix: remove more

* Discard changes to .github/workflows/classic-autogpt-ci.yml

* Update classic-autogpt-docker-ci.yml

* Update classic-autogpt-docker-release.yml

* Update classic-autogpts-ci.yml

* Discard changes to .github/workflows/classic-forge-ci.yml

* Discard changes to .github/workflows/classic-autogpts-ci.yml

* Discard changes to .github/workflows/classic-python-checks.yml

* Discard changes to .github/workflows/repo-pr-label.yml

* Discard changes to .github/workflows/platform-backend-ci.yml

* Update classic-benchmark-ci.yml

* Update classic-frontend-ci.yml
2024-11-15 01:48:00 -06:00
dependabot[bot]
9e4246602d
build(deps): bump peter-evans/repository-dispatch from 2 to 3 (#8609)
Bumps [peter-evans/repository-dispatch](https://github.com/peter-evans/repository-dispatch) from 2 to 3.
- [Release notes](https://github.com/peter-evans/repository-dispatch/releases)
- [Commits](https://github.com/peter-evans/repository-dispatch/compare/v2...v3)

---
updated-dependencies:
- dependency-name: peter-evans/repository-dispatch
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11 19:59:12 +00:00
Reinier van der Leer
c960bd870c
dx: Clean up PR template (#8541)
* dx: Clean up PR template
2024-11-08 17:44:55 +01:00
Aarushi
c0f77c8e7a
ci(platform): Update pipelines to run from infra repo (#8595)
* deploy trigger

* update name

* test path change

* update prod deploy too

* remove envvars

* update prod deploy name
2024-11-07 22:39:59 -06:00
Nicholas Tindle
b08ad973fa
Update codeql.yml (#8500)
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-11-07 02:51:01 +00:00
dependabot[bot]
b223cc0233
build(deps): bump actions/checkout from 2 to 4 (#8553)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 16:21:54 -06:00
dependabot[bot]
7e63a8f94a
build(deps): bump google-github-actions/auth from 1 to 2 (#8468)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 20:28:05 +00:00
dependabot[bot]
456b1e55aa
build(deps): bump google-github-actions/setup-gcloud from 1 to 2 (#8469)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 15:24:01 -05:00
dependabot[bot]
db0f1fa0ec
build(deps): bump peter-evans/create-pull-request from 6 to 7 (#8470)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 15:21:30 -05:00
dependabot[bot]
757fb075b1
build(deps): bump actions/cache from 2 to 4 (#8467)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 15:16:19 -05:00
Aarushi
b2972cc9c6
fix(ci): Fix the prod workload identity pool defined in the prod yaml (#8456)
fix project id
2024-10-28 12:51:51 +00:00
dependabot[bot]
8f5bd40791
build(deps): bump google-github-actions/get-gke-credentials from 1 to 2 (#8437)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 17:53:14 -05:00
dependabot[bot]
19dc2093df
build(deps): bump docker/setup-buildx-action from 1 to 3 (#8436)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 22:47:52 +00:00
dependabot[bot]
ca43f71fa8
build(deps): bump azure/setup-helm from 1 to 4 (#8439)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 22:45:44 +00:00
dependabot[bot]
2fa4d8042c
build(deps): bump helm/chart-testing-action from 2.6.0 to 2.6.1 in the production-dependencies group (#8435)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 17:43:36 -05:00
dependabot[bot]
75f3092ba2
build(deps): bump actions/setup-python from 4 to 5 (#8438)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 22:30:18 +00:00
Aarushi
08a8abeb58
docs(pr_template): Add reminder for config and infra changes (#8430)
add config change reminder
2024-10-24 19:31:53 +01:00
Aarushi
d9b8e0d273
feat(platform/ci): Add prod migrations (#8396)
* ci with workload identity

* temp update

* update name

* wip

* update auth step

* update provider name

* remove audience

* temp set to false

* update registry naming

* update context

* update login

* revert temp updates

* add prod iam and pool

* add release deploy with approval

* use gha default approval behaviour

* add back in release trigger

* add new line

* add prod migrations

* prod migrations without check
2024-10-23 09:25:33 +01:00
Aarushi
404d0638de
feat(platform/ci) Add workflow for deploying to production (#8384)
* ci with workload identity

* temp update

* update name

* wip

* update auth step

* update provider name

* remove audience

* temp set to false

* update registry naming

* update context

* update login

* revert temp updates

* add prod iam and pool

* add release deploy with approval

* use gha default approval behaviour

* add back in release trigger

* add new line
2024-10-23 09:23:54 +01:00
Aarushi
2715b81ff5
feat(platform/ci): Run migrations in ci for dev (#8395)
* run migrations in ci

* update environment

* temp false

* add dev migrations

* remove code change step
2024-10-23 09:21:12 +01:00
Toran Bruce Richards
0c517216df
fix(frontend): Fix broken terms of use link (#8279)
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-10-22 15:39:54 -05:00
Aarushi
30a62f898c
feat(platform/ci) Set up deploys from dev (#8355)
* ci with workload identity

* temp update

* update name

* wip

* update auth step

* update provider name

* remove audience

* temp set to false

* update registry naming

* update context

* update login

* revert temp updates
2024-10-22 13:45:30 +00:00
Nicholas Tindle
2437f58849
ci: create dependabot config (#8372)
* ci: create dependabot

* ci: target the dev branch for dependabot

* ci: group prs

* ci: group updates

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-21 08:05:07 -05:00
Reinier van der Leer
5d4d2486da
ci: Enforce dev as base branch for development (#8369)
* Create repo-pr-enforce-base-branch.yml

* fix quotes

* test

* fix github token

* fix trigger and CLI config

* change back trigger because otherwise I can't test it

* fix the fix

* fix repo selection

* fix perms?

* fix quotes and newlines escaping in message

* Update repo-pr-enforce-base-branch.yml

* grrr escape sequences in bash

* test

* clean up

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-10-17 17:15:15 +01:00
Nicholas Tindle
d6d2820b92
fix(market): agent pagination and search errors (#8336)
* fix(market): agent pagination and search errors

* fix(frontend): search was not paginated

* fix: linting

* feat(market): linting ci

* fix(ci): branch limit name
2024-10-16 20:29:53 +00:00
Aarushi
a2ef456525
fix(ci,platform): Add dev branch trigger to all ci (#8339)
* update ci for dev

* update classic

* remove duplicate dev
2024-10-15 10:57:24 +01:00
Reinier van der Leer
74e677baec
ci(frontend): Enforce consistent yarn.lock + minor DX improvements (#8316)
- ci(frontend): Ensure CI fails if `yarn.lock` is inconsistent with `package.json`
- dx(frontend): Add Prettier check to `lint` script in `package.json`
- dx(frontend): Add `packageManager` to `package.json` for Corepack support
- build(frontend): Use `yarn` consistently in the Dockerfile
2024-10-11 16:51:15 +02:00
Zamil Majdy
daa054c79c
feat(backend): Make Redis connection Sync + Use Redis as Distributed Lock (#8197) 2024-10-07 15:39:32 +07:00
Reinier van der Leer
c64a2d4101
dx(blocks): Auto-label block PRs (#8230)
Automatically apply the `platform/blocks` label to PRs that change files in `backend/blocks/`
2024-09-30 19:52:05 +00:00
Reinier van der Leer
5e2874c315
feat(platform): OAuth support + API key management + GitHub blocks (#8044)
## Config
- For Supabase, the back end needs `SUPABASE_URL`, `SUPABASE_SERVICE_ROLE_KEY`, and `SUPABASE_JWT_SECRET`
- For the GitHub integration to work, the back end needs `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET`
- For integrations OAuth flows to work in local development, the back end needs `FRONTEND_BASE_URL` to generate login URLs with accurate redirect URLs

## REST API
- Tweak output of OAuth `/login` endpoint: add `state_token` separately in response
- Add `POST /integrations/{provider}/credentials` (for API keys)
- Add `DELETE /integrations/{provider}/credentials/{cred_id}`

## Back end
- Add Supabase support to `AppService`
- Add `FRONTEND_BASE_URL` config option, mainly for local development use

### `autogpt_libs.supabase_integration_credentials_store`
- Add `CredentialsType` alias
- Add `.bearer()` helper methods to `APIKeyCredentials` and `OAuth2Credentials`

### Blocks
- Add `CredentialsField(..) -> CredentialsMetaInput`

## Front end
### UI components
- `CredentialsInput` for use on `CustomNode`: allows user to add/select credentials for a service.
  - `APIKeyCredentialsModal`: a dialog for creating API keys
  - `OAuth2FlowWaitingModal`: a dialog to indicate that the application is waiting for the user to log in to the 3rd party service in the provided pop-up window
- `NodeCredentialsInput`: wrapper for `CredentialsInput` with the "usual" interface of node input components
- New icons: `IconKey`, `IconKeyPlus`, `IconUser`, `IconUserPlus`

### Data model
- `CredentialsProvider`: introduces the app-level `CredentialsProvidersContext`, which acts as an application-wide store and cache for credentials metadata.
- `useCredentials` for use on `CustomNode`: uses `CredentialsProvidersContext` and provides node-specific credential data and provider-specific data/functions
- `/auth/integrations/oauth_callback` route to close the loop to the `CredentialsInput` after a user completes sign-in to the external service
- Add `BlockIOCredentialsSubSchema`

### API client
- Add `isAuthenticated` method
- Add methods for integration OAuth flow: `oAuthLogin`, `oAuthCallback`
- Add CRD methods for credentials: `createAPIKeyCredentials`, `listCredentials`, `getCredentials`, `deleteCredentials`
- Add mirrored types `CredentialsMetaResponse`, `CredentialsMetaInput`, `OAuth2Credentials`, `APIKeyCredentials`
- Add GitHub blocks + "DEVELOPER_TOOLS" category
- Add `**kwargs` to `Block.run(..)` signature to support additional kwargs
- Add support for loading blocks from nested modules (e.g. `blocks/github/issues.py`)

#### Executor
- Add strict support for `credentials` fields on blocks
- Fetch credentials for graph execution and pass them down through to the node execution
2024-09-26 00:36:29 +02:00
Nicholas Tindle
46b8f9af0a
feat(builder): scaffold playwright (#8109) 2024-09-24 23:31:38 +00:00
Reinier van der Leer
f109c3e019
ci: Set up customized CodeQL scanning (#8106)
This replaces the dynamically configured CodeQL scanning that we had set up through the GitHub settings.
2024-09-24 18:26:15 +02:00
Reinier van der Leer
679245416f
ci: Reduce PR Status Checker interval (#8137)
Reduce interval from 3 minutes to 30 seconds to reduce CI lag
2024-09-23 13:10:23 +00:00
Reinier van der Leer
c533044cdc
ci(backend): Add Supabase (#8123)
- Set up Supabase in the Backend CI
  - Remove separate Postgres DB setup
- Remove unused MinIO service
- Remove multi-platform stuff
2024-09-23 09:45:20 +00:00
Swifty
ef7cfbb860
refactor: AutoGPT Platform Stealth Launch Repo Re-Org (#8113)
Restructuring the Repo to make it clear the difference between classic autogpt and the autogpt platform:
* Move the "classic" projects `autogpt`, `forge`, `frontend`, and `benchmark` into a `classic` folder
  * Also rename `autogpt` to `original_autogpt` for absolute clarity
* Rename `rnd/` to `autogpt_platform/`
  * `rnd/autogpt_builder` -> `autogpt_platform/frontend`
  * `rnd/autogpt_server` -> `autogpt_platform/backend`
* Adjust any paths accordingly
2024-09-20 16:50:43 +02:00
Aarushi
e3f35d79c7
tweak(.github): Update pr template wording (#8103)
* update pr template wording

* add what and how

* Update .github/PULL_REQUEST_TEMPLATE.md

---------

Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
2024-09-19 12:44:50 +00:00
Aarushi
0040495143
tweak(.github): Update PR template (#8100)
* update PR template

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>

* add note

* typo

---------

Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>
2024-09-19 13:00:16 +01:00
Reinier van der Leer
8fd22bcfd7
feat(server, builder): Implement "STOP" button for graph runs (#7892)
- feat(builder): Add "Stop Run" buttons to monitor and builder
  - Implement additional state management in `useAgentGraph` hook
    - Add "stop" request mechanism
    - Implement execution status tracking using WebSockets
    - Add `isSaving`, `isRunning`, `isStopping` outputs
    - Add `requestStopRun` method
      - Rename `requestSaveRun` to `requestSaveAndRun` for clarity
  - Add needed functionality for the above to `AutoGPTServerAPI` client
    - Add `stopGraphExecution` method
    - Add support for multiple handlers per WebSocket method
    - Fix parsing of timestamps in `execution_event` WebSocket messages
  - Add `IconSquare` from Lucide to `@/components/ui/icons`

- feat(server): Add `POST /graphs/{graph_id}/executions/{graph_exec_id}/stop` route
  - Add `stop_graph_run` method to `AgentServer`

- feat(server): Add `cancel_execution` method to `ExecutionManager`
  - Replace node executor `ProcessPoolExecutor` by `multiprocessing.Pool` (which has a `terminate()` method)
    - Remove now unnecessary `Executor.wait_future(..)` method
  - Add `get_graph_execution(..)` in `.data.execution`

- fix(server): Reduce number of node executors to 5 per graph executor
  This is necessary because `multiprocessing.Pool` spawns its workers on init, instead of based on demand like `ProcessPoolExecutor` does

- dx(server): Improve debug logging in `ExecutionManager`
- ci(server): Add debug logging mode to CI Pytest step

### Other improvements
Server:
- Improve output type of `ExecutionManager.add_execution(..)`
- Renamed a few things in `.server.rest_api` for consistency

Front end:
- Improved typing in `AutoGPTServerAPI` client
2024-09-05 14:42:28 +02:00
Aarushi
5000aa7ee0
tweak(rnd,docker) Remove SQLite (#7966)
* move migrations, update networking and dockignore

* update docs

* remove sqlite from ci

* remove schema linting checks

* fix formatting

* remove schema linting

* add test script

* formatting and linting

* stop pg not down

* seperate test db

* diff port

* remove duplicate
2024-09-04 10:18:57 +01:00