Merge pull request #30 from BBai-Tips/staging

TLS support and Windows compat

CHANGELOG.md

@@ -9,6 +9,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 
 
+## [0.0.16-beta] - 2024-09-27
+
+### Changed
+
+- Fixed 'git not found' error when running init
+- Updated `bbai` strings to `bbai.exe` for Windows
+- Fixes for Windows compat
+- Added config option for API and BUI to listen with TLS
+- Updated CLI and BUI to initiate TLS connections if useTls is set
+- Added certificate generation to init process
+- Updated docs
+
+
 ## [0.0.15-beta] - 2024-09-25
 
 ### Changed

CONVENTIONS.md

@@ -48,11 +48,11 @@ It supports vector embeddings for code and text chunks from local repositories,
 
 ## Glossary of Terms
 
-- **Conversation**: An ongoing interaction between a user and the bbai system, which may include multiple statements and turns.
-- **Statement**: A single request or input from the user to the bbai system.
-- **Turn**: A single request-response cycle between bbai and the LLM.
-- **Session**: The entire duration of a user's interaction with bbai, which may include multiple conversations.
-- **Project**: The collection of files and resources that bbai is working with.
+- **Conversation**: An ongoing interaction between a user and the BBai system, which may include multiple statements and turns.
+- **Statement**: A single request or input from the user to the BBai system.
+- **Turn**: A single request-response cycle between BBai and the LLM.
+- **Session**: The entire duration of a user's interaction with BBai, which may include multiple conversations.
+- **Project**: The collection of files and resources that BBai is working with.
 - **Tool**: A specific function or capability that the LLM can use to perform actions or retrieve information.
 - **Patch**: A set of changes to be applied to a file.
 - **Commit**: A saved state of the project in the version control system.
@@ -102,7 +102,7 @@ Key API Endpoints:
 - Ensure cross-platform compatibility
 - Use descriptive names for CLI-specific components
 - Prefix CLI-specific types with `CLI`
-- The CLI tool is named `bbai` (lowercase)
+- The CLI tool is named `bbai` (lowercase) (`bbai.exe` for Windows)
 - The API server is referred to as 'api'
 - The Browser User Interface is referred to as 'bui'
 - The future Desktop User Interface will be referred to as 'dui'

INSTALL.md

@@ -13,8 +13,24 @@ Before using BBai, ensure you have the following:
 1. An Anthropic API key (Note: This is different from your Anthropic chat console login. You can create an API key at https://console.anthropic.com/settings/keys)
 2. [Git](https://git-scm.com/) (latest stable version, recommended but optional)
 3. [ctags](https://github.com/universal-ctags/ctags) (optional, enhances project understanding)
+4. Either `mkcert` or `openssl` for TLS certificate generation (required for proper operation)
 
-Git and ctags can be easily installed using package managers like Homebrew on macOS or apt on Linux. While Git is optional, it's highly recommended for optimal use of BBai.
+Git, ctags, and mkcert can be easily installed using package managers like Homebrew on macOS, Chocolatey on Windows, or apt on Linux. While Git is optional, it's highly recommended for optimal use of BBai.
+
+To install `mkcert`:
+- On Windows: `choco install mkcert`
+- On macOS: `brew install mkcert`
+- On Linux: Follow the instructions at https://github.com/FiloSottile/mkcert#linux
+
+Note: TLS certificates are required for proper operation of BBai. The initialization process will automatically generate the necessary certificates using either `mkcert` or `openssl` if they are installed. If neither is available, an error will be generated explaining how to install them.
+
+For technical users: Any valid TLS certificate can be used. BBai provides four config options for custom certificates:
+- `api.tlsKeyFile`: File path to the TLS key
+- `api.tlsCertFile`: File path to the TLS certificate
+- `api.tlsKeyPem`: Inlined PEM content of the TLS key
+- `api.tlsCertPem`: Inlined PEM content of the TLS certificate
+
+Use either file paths or inlined PEM content, not both.
 
 ## Installation Methods
 
@@ -33,11 +49,6 @@ This script will:
 
 Note: You may be prompted for your password to install the binaries in `/usr/local/bin`. This is necessary to make BBai accessible system-wide.
 
-The installation process uses color-coded output for better readability:
-- Yellow: Information and progress messages
-- Red: Important notes or warnings
-- Green: Success messages
-
 ### Option 2: Windows Installer
 
 For Windows users, we provide an MSI installer for easy installation:
@@ -63,8 +74,8 @@ For advanced users who prefer manual installation:
 
 1. Go to the [BBai Releases page](https://github.com/BBai-Tips/bbai/releases) on GitHub.
 2. Download the appropriate package for your operating system and architecture:
-   - For Linux: `bbai-x86_64-unknown-linux-gnu.tar.gz` or `bbai-aarch64-unknown-linux-gnu.tar.gz`
    - For macOS: `bbai-x86_64-apple-darwin.tar.gz` or `bbai-aarch64-apple-darwin.tar.gz`
+   - For Linux: `bbai-x86_64-unknown-linux-gnu.tar.gz` or `bbai-aarch64-unknown-linux-gnu.tar.gz`
    - For Windows: `bbai-x86_64-pc-windows-msvc.zip`
 3. Extract the downloaded package:
    - For .tar.gz files (Linux and macOS):
@@ -111,7 +122,13 @@ After installation, navigate to the project directory where you want to use BBai
 bbai init
 ```
 
-This will create a `.bbai/config.yaml` file in your project directory.
+On Windows:
+
+```
+bbai.exe init
+```
+
+This will create a `.bbai/config.yaml` file in your project directory and generate the necessary TLS certificates for secure operation. If `mkcert` or `openssl` is not available, you will receive an error message with instructions on how to install them.
 
 ## Setting Up Your Anthropic API Key
 
@@ -144,6 +161,13 @@ bbai --help
 bbai-api --help
 ```
 
+On Windows:
+
+```
+bbai.exe --help
+bbai-api.exe --help
+```
+
 These commands should display the help information for BBai and its API.
 
 ## Usage

README.md

@@ -86,8 +86,16 @@ Before using BBai, ensure you have the following:
 1. An Anthropic API key (Note: This is different from your Anthropic chat console login. You'll need to create an API key at https://console.anthropic.com/settings/keys)
 2. [Git](https://git-scm.com/downloads) (latest stable version, recommended but optional)
 3. [ctags](https://github.com/universal-ctags/ctags) (optional, enhances project understanding)
+4. Either `mkcert` or `openssl` for TLS certificate generation (required for proper operation)
 
-Git and ctags can be easily installed using package managers like Homebrew on macOS or apt on Linux. While Git is optional, it's highly recommended for optimal use of BBai.
+To install `mkcert`:
+- On Windows: `choco install mkcert`
+- On macOS: `brew install mkcert`
+- On Linux: Follow the instructions at https://github.com/FiloSottile/mkcert#linux
+
+Note for technical users: Any valid TLS certificate can be used. BBai provides four config options for custom certificates: `api.tlsKeyFile`, `api.tlsCertFile`, `api.tlsKeyPem`, `api.tlsCertPem`. Use either file paths or inlined PEM content, not both.
+
+Git, ctags, and mkcert can be easily installed using package managers like Homebrew on macOS, Chocolatey on Windows, or apt on Linux. While Git is optional, it's highly recommended for optimal use of BBai. TLS certificates are required for proper operation, and will be automatically generated during the initialization process if not provided.
 
 ### Installation
 
@@ -119,6 +127,8 @@ After installation, you can start using BBai as follows:
    bbai chat
    ```
 
+On Windows, use `bbai.exe` instead of `bbai`.
+
 ## Documentation
 
 For detailed information on how to use BBai, please refer to our documentation:
@@ -183,8 +193,8 @@ It's compatible with projects using various programming languages and text-based
 We value your input and are here to help you get the most out of BBai:
 
 - For bug reports or feature requests, please [open an issue](https://github.com/BBai-Tips/bbai/issues) on our GitHub repository.
-- For general questions or discussions, join our [community forum](https://github.com/BBai-Tips/bbai/discussions) (link to be updated).
-- For more immediate support, reach out to us via email at [email protected] (to be set up).
+- For general questions or discussions, join our [community forum](https://github.com/BBai-Tips/bbai/discussions).
+- For more immediate support, reach out to us via email at [email protected].
 
 Your feedback is crucial in shaping the future of BBai!
 

TODO.md

@@ -19,15 +19,15 @@
 2. Implement a way to "end" conversation which merges branch back to original
 
 ## Task Management
-1. Create an 'add task' tool allowing Claude to give bbai a list of tasks to complete
+1. Create an 'add task' tool allowing Claude to give BBai a list of tasks to complete
 2. Implement each task as a new conversation to complete the task
 
 ## System Prompt and Project Info
 1. Update system prompt to give a clear explanation about using project-info to choose files rather than search_files
 2. Move project-info towards the beginning of the system prompt
 3. Create a high-level system prompt template to combine baseSystem with project info and files added
-4. Update system prompt to further clarify that assistant is talking to both bbai and user:
-   - Responses to tool use (e.g., "Patch applied successfully") should be directed to bbai and wrapped in tags for parsing
+4. Update system prompt to further clarify that assistant is talking to both BBai and user:
+   - Responses to tool use (e.g., "Patch applied successfully") should be directed to BBai and wrapped in tags for parsing
    - Everything not inside <bbai> tags will be shown to user as part of the conversation
    - 'User' message showing 'tool result' should be clearly separate from rest of the conversation
 
@@ -66,7 +66,7 @@
 
 ## For CHANGELOG
 - Implement a repoInfo persistence solution
-- Create an 'add task' tool allowing Claude to give bbai a list of tasks to complete
+- Create an 'add task' tool allowing Claude to give BBai a list of tasks to complete
 
 ## Completed Tasks
 1. ✓ `searchFiles` is now using exclude patterns read from the file instead of file names for exclude pattern
@@ -98,7 +98,7 @@
     - Allow users to tail the chat log directly for as-is viewing
     - Use `bbai logs` for fancy formatting
     - Make `bbai logs` check the TERM width and use that for max width
-22. ✓ Add 'bbai restart' command for API (can just do a stop/start)
+22. ✓ Add `bbai restart` command for API (can just do a stop/start)
 23. ✓ Implement tool manager class
 24. ✓ Add websocket mode to get live updates between terminal and API
 25. ✓ Fix exclude args not being respected for file listing in `fileHandling.utils.ts`

api/deno.jsonc

@@ -1,6 +1,6 @@
 {
 	"name": "bbai-api",
-	"version": "0.0.15-beta",
+	"version": "0.0.16-beta",
 	"exports": "./src/main.ts",
 	"tasks": {
 		"start": "deno run --allow-read --allow-write --allow-run --allow-net --allow-env src/main.ts",

api/prompts/interview.md

@@ -1,6 +1,6 @@
 ---
 name: Interview Prompt
-description: Default interview prompt for bbai
+description: Default interview prompt for BBai
 version: 1.0.0
 ---
 Act as an expert software project manager, ask me one question at a time until you have enough information to write a detailed statement of work according to all the industry best practices. For each question you ask make sure to inform me of the options and their respective trade offs.

api/prompts/system.md

@@ -1,9 +1,9 @@
 ---
 name: System Prompt
-description: Default system prompt for bbai
+description: Default system prompt for BBai
 version: 1.0.0
 ---
-You are an AI assistant named bbai, designed to help with various text-based projects. Your capabilities include:
+You are an AI assistant named BBai, designed to help with various text-based projects. Your capabilities include:
 
 1. Analyzing and modifying programming code in any language
 2. Reviewing and enhancing documentation and prose

api/src/main.ts

@@ -1,4 +1,5 @@
 import { Application } from '@oak/oak';
+import type { ListenOptions, ListenOptionsTls } from '@oak/oak';
 import oak_logger from 'oak_logger';
 import { parseArgs } from '@std/cli';
 import { oakCors } from 'cors';
@@ -7,27 +8,32 @@ import { ConfigManager } from 'shared/configManager.ts';
 import router from './routes/routes.ts';
 import { logger } from 'shared/logger.ts';
 import type { BbAiState } from 'api/types.ts';
+import { readFromBbaiDir, readFromGlobalConfigDir } from 'shared/dataDir.ts';
+import { apiFileLogger } from './utils/fileLogger.ts';
 
-const fullConfig = await ConfigManager.fullConfig(Deno.cwd());
-const redactedFullConfig = await ConfigManager.redactedFullConfig(Deno.cwd());
-const { environment, apiHostname, apiPort } = fullConfig.api || {};
+// CWD is set by `bbai` in Deno.Command, or implicitly set by user if calling bbai-api directly
+const startDir = Deno.cwd();
+const fullConfig = await ConfigManager.fullConfig(startDir);
+const redactedFullConfig = await ConfigManager.redactedFullConfig(startDir);
+const { environment, apiHostname, apiPort, apiUseTls } = fullConfig.api;
 
 // Parse command line arguments
 const args = parseArgs(Deno.args, {
 	string: ['log-file', 'port', 'hostname'],
 	boolean: ['help', 'version'],
-	alias: { h: 'help', V: 'version', v: 'version', p: 'port', H: 'hostname', l: 'log-file' },
+	alias: { h: 'help', V: 'version', v: 'version', p: 'port', H: 'hostname', t: 'useTls', l: 'log-file' },
 });
 
 if (args.help) {
 	console.log(`
-Usage: bbai-api [options]
+Usage: ${fullConfig.bbaiApiExeName} [options]
 
 Options:
   -h, --help                Show this help message
   -V, --version             Show version information
   -H, --hostname <string>   Specify the hostname to run the API server (default: ${apiHostname})
   -p, --port <number>       Specify the port to run the API server (default: ${apiPort})
+  -t, --useTls <boolean>    Specify whether the API server should use TLS (default: ${apiUseTls})
   -l, --log-file <file>     Specify a log file to write output
   `);
 	Deno.exit(0);
@@ -38,37 +44,17 @@ if (args.version) {
 	Deno.exit(0);
 }
 
+// Redirect console.log and console.error to the log file
 const apiLogFile = args['log-file'];
+if (apiLogFile) await apiFileLogger(apiLogFile);
+
 const customHostname = args.hostname ? args.hostname : apiHostname;
-const customPort = args.port ? parseInt(args.port, 10) : apiPort;
+const customPort: number = args.port ? parseInt(args.port, 10) : apiPort as number;
+const customUseTls: boolean = typeof args.useTls !== 'undefined'
+	? (args.useTls === 'true' ? true : false)
+	: !!apiUseTls;
 //console.debug(`BBai API starting at ${customHostname}:${customPort}`);
 
-if (apiLogFile) {
-	// Redirect console.log and console.error to the log file
-	const consoleFunctions = ['log', 'debug', 'info', 'warn', 'error'];
-
-	const apiLogFileStream = await Deno.open(apiLogFile, { write: true, create: true, append: true });
-	const encoder = new TextEncoder();
-
-	consoleFunctions.forEach((funcName) => {
-		(console as any)[funcName] = (...args: any[]) => {
-			const timestamp = new Date().toISOString();
-			const prefix = funcName === 'log' ? '' : `[${funcName.toUpperCase()}] `;
-			const message = `${timestamp} ${prefix}${
-				args.map((arg) => typeof arg === 'string' ? arg : JSON.stringify(arg)).join(' ')
-			}\n`;
-			apiLogFileStream.write(encoder.encode(message));
-		};
-	});
-
-	// Redirect Deno.stderr to the log file
-	const originalStderrWrite = Deno.stderr.write;
-	Deno.stderr.write = (p: Uint8Array): Promise<number> => {
-		apiLogFileStream.write(p);
-		return originalStderrWrite.call(Deno.stderr, p);
-	};
-}
-
 const app = new Application<BbAiState>();
 
 app.use(oak_logger.logger);
@@ -95,8 +81,20 @@ app.addEventListener('error', (evt: ErrorEvent) => {
 });
 
 if (import.meta.main) {
+	let listenOpts: ListenOptions = { hostname: customHostname, port: customPort };
+	if (customUseTls) {
+		const cert = fullConfig.api.tlsCertPem ||
+			await readFromBbaiDir(startDir, fullConfig.api.tlsCertFile || 'localhost.pem') ||
+			await readFromGlobalConfigDir(fullConfig.api.tlsCertFile || 'localhost.pem') || '';
+		const key = fullConfig.api.tlsKeyPem ||
+			await readFromBbaiDir(startDir, fullConfig.api.tlsKeyFile || 'localhost-key.pem') ||
+			await readFromGlobalConfigDir(fullConfig.api.tlsKeyFile || 'localhost-key.pem') || '';
+
+		listenOpts = { ...listenOpts, secure: true, cert, key } as ListenOptionsTls;
+	}
+
 	try {
-		await app.listen({ hostname: customHostname, port: customPort });
+		await app.listen(listenOpts);
 	} catch (error) {
 		logger.error(`Failed to start server: ${error.message}`);
 		logger.error(`Stack trace: ${error.stack}`);

api/src/prompts/defaultPrompts.ts

@@ -18,7 +18,7 @@ interface Prompt {
 export const system: Prompt = {
 	metadata: {
 		name: 'System Prompt',
-		description: 'Default system prompt for bbai',
+		description: 'Default system prompt for BBai',
 		version: '1.0.0',
 	},
 	getContent: async ({ userDefinedContent = '', fullConfig }) => {
@@ -46,7 +46,7 @@ export const system: Prompt = {
 		  5. Working with HTML, SVG, and various markup languages
 		  6. Handling configuration files and data formats (JSON, YAML, etc.)
 	
-		  You are facilitating a conversation between "bbai" (an AI-powered writing assistant) and the user named "${myPersonsName}". All conversation messages will be labeled as either 'assistant' or 'user'. The 'user' messages will contain instructions from both "bbai" and "${myPersonsName}". You should respect instructions from both "bbai" and "${myPersonsName}" but always prioritize instructions or comments from ${myPersonsName}. When addressing the user, refer to them as "${myPersonsName}". When providing instructions for the writing assistant, refer to it as "bbai". Wrap instructions for "bbai" with <bbai> XML tags. Always prefer using a tool rather than writing instructions to "bbai".
+		  You are facilitating a conversation between "BBai" (an AI-powered writing assistant) and the user named "${myPersonsName}". All conversation messages will be labeled as either 'assistant' or 'user'. The 'user' messages will contain instructions from both "BBai" and "${myPersonsName}". You should respect instructions from both "BBai" and "${myPersonsName}" but always prioritize instructions or comments from ${myPersonsName}. When addressing the user, refer to them as "${myPersonsName}". When providing instructions for the writing assistant, refer to it as "BBai". Wrap instructions for "BBai" with <bbai> XML tags. Always prefer using a tool rather than writing instructions to "BBai".
 	
 		  In each conversational turn, you will begin by thinking about your response. Once you're done, you will write a user-facing response for "${myPersonsName}". It's important to place all user-facing conversational responses in <reply></reply> XML tags to make them easy to parse.
 	

api/src/routes/routes.ts

@@ -12,7 +12,7 @@ router
 	 * /:
 	 *   get:
 	 *     summary: API root
-	 *     description: Returns a welcome message for the bbai API and provides a link to the API documentation
+	 *     description: Returns a welcome message for the BBai API and provides a link to the API documentation
 	 *     responses:
 	 *       200:
 	 *         description: Successful response with welcome message and documentation link

api/src/routes/swaggerRouter.ts

@@ -30,7 +30,7 @@ const swaggerSpec = swaggerJsdoc(options);
  * /api-docs/openapi.json:
  *   get:
  *     summary: Get OpenAPI specification
- *     description: Returns the OpenAPI specification for the bbai API in JSON format
+ *     description: Returns the OpenAPI specification for the BBai API in JSON format
  *     responses:
  *       200:
  *         description: Successful response with OpenAPI specification
@@ -45,7 +45,7 @@ const swaggerSpec = swaggerJsdoc(options);
  * /api-docs/swagger.json:
  *   get:
  *     summary: Get Swagger specification
- *     description: Returns the Swagger specification for the bbai API in JSON format
+ *     description: Returns the Swagger specification for the BBai API in JSON format
  *     responses:
  *       200:
  *         description: Successful response with Swagger specification