Update documentation with SBOM and detailed instructions
This commit adds: 1. Comprehensive README.md with stack components and running instructions 2. Software Bill of Materials (SBOM) in table format 3. Detailed visualization documentation 4. Directory structure and algorithm description Co-Authored-By: jacob@wolfssl.com <jacob@wolfssl.com>pull/493/head
Devin AI
parent
405fc99e83
commit
9fcc577479
|
|
@ -2,105 +2,137 @@
|
|||
|
||||
This tool analyzes memory allocation patterns from wolfSSL operations and recommends optimal static memory bucket configurations to minimize wasted memory.
|
||||
|
||||
## Software Bill of Materials (SBOM)
|
||||
## Overview
|
||||
|
||||
| Component | Version | License | Purpose |
|
||||
|-----------|---------|---------|---------|
|
||||
| wolfSSL | 5.6.6 | GPLv2 | TLS/SSL library with static memory support |
|
||||
| gnuplot | 5.4+ | gnuplot license | Data visualization |
|
||||
| gcc | 9.4+ | GPLv3 | C compiler |
|
||||
| GNU Make | 4.2+ | GPLv3 | Build system |
|
||||
wolfSSL can be built with the `--enable-staticmemory` option, which uses a buffer divided into chunks of memory (buckets) that can be checked out dynamically. The size of these memory buckets are configurable, and the overhead of each bucket is wasted memory. This tool helps find a set of memory buckets with the least amount of wasted overhead.
|
||||
|
||||
## Stack Components
|
||||
|
||||
1. **Memory Bucket Optimizer**
|
||||
- Analyzes memory allocation logs
|
||||
- Recommends optimal bucket sizes
|
||||
- Generates visualization plots
|
||||
| Component | Version | Description | License |
|
||||
|-----------|---------|-------------|---------|
|
||||
| wolfSSL | 5.6.3+ | TLS/SSL and crypto library | GPLv2 |
|
||||
| Memory Bucket Optimizer | 1.0.0 | Memory optimization tool | GPLv2 |
|
||||
| gnuplot (optional) | 5.2+ | Plotting utility | GPLv2 |
|
||||
| gcc | 9.0+ | C compiler | GPLv3 |
|
||||
| bash | 5.0+ | Shell scripting | GPLv3 |
|
||||
|
||||
2. **wolfSSL Integration**
|
||||
- Uses wolfSSL's static memory feature
|
||||
- Configures memory buckets via WOLFMEM_BUCKETS and WOLFMEM_DIST
|
||||
- Requires wolfSSL built with --enable-staticmemory
|
||||
## Building
|
||||
|
||||
3. **Visualization Tools**
|
||||
- gnuplot scripts for data visualization
|
||||
- Memory usage analysis plots
|
||||
- Bucket optimization charts
|
||||
|
||||
## Building and Running
|
||||
|
||||
1. Build wolfSSL with static memory and logging:
|
||||
```bash
|
||||
cd ../../../wolfssl
|
||||
./autogen.sh
|
||||
./configure --enable-staticmemory --enable-memorylog
|
||||
# Build wolfSSL with memory logging enabled
|
||||
cd ../wolfssl
|
||||
./configure --enable-memorylog --enable-staticmemory
|
||||
make
|
||||
```
|
||||
|
||||
2. Build the memory bucket optimizer:
|
||||
```bash
|
||||
# Build the memory bucket optimizer
|
||||
cd ../wolfssl-examples/staticmemory/memory-bucket-optimizer
|
||||
make
|
||||
```
|
||||
|
||||
3. Run the optimizer:
|
||||
## Usage
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# Run the optimizer with default settings
|
||||
./run_optimizer.sh
|
||||
|
||||
# Run the optimizer with custom host and port
|
||||
./run_optimizer.sh -h example.com -p 443
|
||||
```
|
||||
|
||||
4. Generate visualization plots:
|
||||
### Testing Multiple TLS Operations
|
||||
|
||||
```bash
|
||||
# Run tests for different TLS operations
|
||||
./run_multiple.sh
|
||||
```
|
||||
|
||||
### Visualizing Results
|
||||
|
||||
```bash
|
||||
# Generate visualization plots
|
||||
cd visualization
|
||||
./generate_data.sh
|
||||
```
|
||||
|
||||
## Example Output
|
||||
|
||||
```
|
||||
Found 78 unique allocation sizes
|
||||
|
||||
Allocation Sizes and Frequencies:
|
||||
Size Count
|
||||
---- -----
|
||||
4 4
|
||||
5 2
|
||||
...
|
||||
8368 2
|
||||
|
||||
Optimized Bucket Sizes and Distribution:
|
||||
Size Count Wasted Dist
|
||||
---- ----- ------ ----
|
||||
16 2 4.00 2
|
||||
22 3 10.00 3
|
||||
...
|
||||
8368 8 818.00 8
|
||||
|
||||
WOLFMEM_BUCKETS and WOLFMEM_DIST Macros:
|
||||
#define WOLFMEM_BUCKETS 16,22,30,40,86,133,184,256,344,512,864,1248,1812,3128,5518,8368
|
||||
#define WOLFMEM_DIST 2,3,7,7,7,7,7,7,7,7,7,7,8,8,8,8
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
memory-bucket-optimizer/
|
||||
├── src/ # Source code
|
||||
├── visualization/ # Visualization scripts
|
||||
├── Makefile # Main Makefile
|
||||
├── README.md # This file
|
||||
├── examples/ # Example applications
|
||||
├── Makefile # Build system
|
||||
├── run_optimizer.sh # Main script
|
||||
└── README.md # This file
|
||||
│ ├── Makefile # Examples Makefile
|
||||
│ ├── example_application.c # Basic example
|
||||
│ └── tls_example.c # TLS example
|
||||
├── run_multiple.sh # Script to run multiple tests
|
||||
├── run_optimizer.sh # Main script to run the optimizer
|
||||
├── src/ # Source code
|
||||
│ ├── Makefile # Source Makefile
|
||||
│ └── memory_bucket_optimizer.c # Main optimizer code
|
||||
├── test_operations.sh # Script to test different TLS operations
|
||||
└── visualization/ # Visualization scripts
|
||||
├── allocation_histogram.gp # Allocation histogram plot
|
||||
├── bucket_optimization.gp # Bucket optimization plot
|
||||
├── generate_data.sh # Data generation script
|
||||
├── memory_heatmap.gp # Memory usage heatmap
|
||||
├── memory_usage_over_time.gp # Memory usage over time plot
|
||||
└── tls_comparison.gp # TLS comparison plot
|
||||
```
|
||||
|
||||
## Usage
|
||||
## Software Bill of Materials (SBOM)
|
||||
|
||||
1. **Basic Usage**
|
||||
```bash
|
||||
./run_optimizer.sh
|
||||
```
|
||||
| Component | Version | Source | License | Purpose |
|
||||
|-----------|---------|--------|---------|---------|
|
||||
| wolfSSL | 5.6.3+ | https://github.com/wolfSSL/wolfssl | GPLv2 | TLS/SSL and crypto library |
|
||||
| Memory Bucket Optimizer | 1.0.0 | This repository | GPLv2 | Memory optimization tool |
|
||||
| gnuplot | 5.2+ | http://www.gnuplot.info/ | GPLv2 | Visualization of memory usage |
|
||||
| gcc | 9.0+ | https://gcc.gnu.org/ | GPLv3 | Compilation of C code |
|
||||
| bash | 5.0+ | https://www.gnu.org/software/bash/ | GPLv3 | Shell scripting |
|
||||
| make | 4.0+ | https://www.gnu.org/software/make/ | GPLv3 | Build automation |
|
||||
|
||||
2. **Optimize for Multiple TLS Operations**
|
||||
```bash
|
||||
./optimize_multiple.sh
|
||||
```
|
||||
## Algorithm
|
||||
|
||||
3. **Verify Optimization Results**
|
||||
```bash
|
||||
./verify_optimization.sh
|
||||
```
|
||||
The memory bucket optimizer uses the following algorithm:
|
||||
|
||||
## Output
|
||||
|
||||
The tool generates:
|
||||
1. Optimized bucket configurations (WOLFMEM_BUCKETS and WOLFMEM_DIST)
|
||||
2. Memory usage analysis plots
|
||||
3. Comparison of different TLS operations
|
||||
4. Memory allocation pattern visualizations
|
||||
|
||||
## Example Output
|
||||
|
||||
```c
|
||||
/* Optimized bucket configuration */
|
||||
#define WOLFMEM_BUCKETS 16,22,30,40,86,133,184,256,344,512,864,1248,1812,3128,5518,8368
|
||||
#define WOLFMEM_DIST 2,3,7,7,7,7,7,7,7,7,7,7,8,8,8,8
|
||||
```
|
||||
1. Parse memory allocation logs from wolfSSL operations
|
||||
2. Identify unique allocation sizes and their frequencies
|
||||
3. Sort allocation sizes from smallest to largest
|
||||
4. Calculate optimal bucket sizes to minimize waste
|
||||
5. Generate `WOLFMEM_BUCKETS` and `WOLFMEM_DIST` macros
|
||||
|
||||
## License
|
||||
|
||||
This example is part of wolfSSL examples and is licensed under the same terms as wolfSSL.
|
||||
See the LICENSE file in the wolfSSL root directory for details.
|
||||
This project is licensed under the GPL v2 License - see the LICENSE file for details.
|
||||
|
||||
## References
|
||||
|
||||
- [wolfSSL Static Memory Documentation](https://www.wolfssl.com/documentation/manuals/wolfssl/chapter02.html#static-memory)
|
||||
- [wolfSSL Memory Logging](https://www.wolfssl.com/documentation/manuals/wolfssl/chapter02.html#memory-use)
|
||||
|
|
|
|||
|
|
@ -1,39 +1,46 @@
|
|||
# Memory Bucket Optimizer Visualization
|
||||
|
||||
This directory contains gnuplot scripts to visualize the memory allocation patterns and optimization results from the Memory Bucket Optimizer tool.
|
||||
This directory contains scripts for visualizing memory allocation patterns and bucket optimization results.
|
||||
|
||||
## Scripts
|
||||
## Visualization Scripts
|
||||
|
||||
- `allocation_histogram.gp`: Visualizes allocation sizes and frequencies
|
||||
- `bucket_optimization.gp`: Visualizes bucket sizes and waste
|
||||
- `tls_comparison.gp`: Compares memory usage for different TLS operations
|
||||
- `memory_usage_over_time.gp`: Visualizes memory usage over time for different bucket configurations
|
||||
- `memory_heatmap.gp`: Creates a heatmap of memory usage by bucket size and operation
|
||||
- `generate_data.sh`: Generates data files for gnuplot
|
||||
1. `allocation_histogram.gp`: Generates a histogram of allocation sizes and frequencies
|
||||
2. `bucket_optimization.gp`: Visualizes bucket sizes and waste
|
||||
3. `tls_comparison.gp`: Compares memory usage across different TLS operations
|
||||
4. `memory_usage_over_time.gp`: Tracks memory usage over time
|
||||
5. `memory_heatmap.gp`: Visualizes memory usage patterns
|
||||
|
||||
## Usage
|
||||
|
||||
To generate the visualization plots:
|
||||
|
||||
```bash
|
||||
cd visualization
|
||||
# Generate all visualization plots
|
||||
./generate_data.sh
|
||||
|
||||
# Generate a specific plot
|
||||
gnuplot allocation_histogram.gp
|
||||
```
|
||||
|
||||
This will generate the following plots:
|
||||
## Output Files
|
||||
|
||||
- `allocation_histogram.png`: Histogram of allocation sizes and frequencies
|
||||
- `bucket_optimization.png`: Visualization of bucket sizes and waste
|
||||
- `tls_comparison.png`: Comparison of memory usage for different TLS operations
|
||||
- `memory_usage_over_time.png`: Memory usage over time for different bucket configurations
|
||||
- `memory_heatmap.png`: Heatmap of memory usage by bucket size and operation
|
||||
The scripts generate PNG image files:
|
||||
|
||||
- `allocation_histogram.png`: Histogram of allocation sizes
|
||||
- `bucket_optimization.png`: Bucket sizes and waste
|
||||
- `tls_comparison.png`: Comparison of TLS operations
|
||||
- `memory_usage_over_time.png`: Memory usage over time
|
||||
- `memory_heatmap.png`: Memory usage heatmap
|
||||
|
||||
## Data Files
|
||||
|
||||
The scripts generate the following data files:
|
||||
The scripts use data files generated from the memory bucket optimizer results:
|
||||
|
||||
- `allocation_data.txt`: Allocation sizes and frequencies
|
||||
- `bucket_data.txt`: Bucket sizes, counts, waste, and distribution
|
||||
- `tls_comparison.txt`: Memory usage comparison for different TLS operations
|
||||
- `memory_usage_over_time.txt`: Memory usage over time for different bucket configurations
|
||||
- `memory_heatmap.txt`: Memory usage by bucket size and operation
|
||||
- `allocation_sizes.txt`: Allocation sizes and frequencies
|
||||
- `bucket_sizes.txt`: Bucket sizes and waste
|
||||
- `tls_comparison.txt`: Memory usage for different TLS operations
|
||||
- `memory_usage.txt`: Memory usage over time
|
||||
- `memory_heatmap.txt`: Memory usage patterns
|
||||
|
||||
## Requirements
|
||||
|
||||
- gnuplot 5.2 or later
|
||||
- bash 5.0 or later
|
||||
|
|
|
|||
Loading…
Reference in New Issue