Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix xvc file track memory issue #260

Merged
merged 262 commits into from
Sep 9, 2024
Merged
Changes from 1 commit
Commits
Show all changes
262 commits
Select commit Hold shift + click to select a range
5fbfdcb
Rename remote to storage in various files
iesahin Jun 14, 2024
0189b38
update
iesahin Jun 17, 2024
4df7a67
Merge ; commit '0189b38859649424af28b321740041086d026541'
iesahin Jun 29, 2024
e6b17bc
fix compile error
iesahin Jul 2, 2024
4653b69
TIDY - Rename XvcConfigInitParams to XvcConfigParams
iesahin Jul 3, 2024
4acf6f2
derive Clone for CLI structs
iesahin Jul 11, 2024
2471651
minor fix
iesahin Jul 11, 2024
ec37735
Renamed ITEMS env variables
iesahin Jul 13, 2024
7e20029
bump version
iesahin Jul 13, 2024
0d85aa6
added CRLF at the end of DONE
iesahin Jul 14, 2024
f365a42
fix the space at the end of out
iesahin Jul 14, 2024
216a419
fix space at the end
iesahin Jul 14, 2024
a901c83
bump version to release
iesahin Jul 15, 2024
5da8604
Merge remote-tracking branch 'origin/main' into develop
iesahin Jul 15, 2024
d9a19eb
fix regex
iesahin Jul 15, 2024
f4ffa53
file-copy
iesahin Jul 15, 2024
5a21dcd
Update docs
iesahin Jul 15, 2024
b7fec60
update CRLF in pipeline
iesahin Jul 15, 2024
cac89d0
Update docs
iesahin Jul 15, 2024
6320123
move generic to bottom
iesahin Jul 18, 2024
96ede58
added xvc-pipeline-step-dependency-sqlite file
iesahin Jul 18, 2024
692e838
Add rusqlite dependency
iesahin Jul 22, 2024
6839540
Updated dependencies
iesahin Jul 22, 2024
4959af5
crate upgrades
iesahin Jul 22, 2024
e9e2ebc
Updated to v0.6.9-a.0
iesahin Jul 22, 2024
bb540a6
Update track
iesahin Jul 22, 2024
b9966f8
update
iesahin Jul 22, 2024
8fb8ff9
cached type is now ty
iesahin Jul 22, 2024
09e7f77
fix import
iesahin Jul 22, 2024
4e31457
import fixes
iesahin Jul 22, 2024
3bed62b
Fix async API change
iesahin Jul 22, 2024
8cc2c08
one more result check
iesahin Jul 22, 2024
6cc63b5
update docs
iesahin Jul 22, 2024
5cac166
move and recheck
iesahin Jul 22, 2024
60a210a
ws
iesahin Jul 22, 2024
38fd54d
Added implementation for SqliteDependency
iesahin Jul 24, 2024
7960a60
added missing branches
iesahin Jul 26, 2024
90a85de
Fix for args
iesahin Jul 26, 2024
8315574
handler update
iesahin Jul 26, 2024
6bd5686
unnecessary let
iesahin Jul 26, 2024
dea58d3
Fix minor errors
iesahin Jul 26, 2024
6e38ce7
xvc file track fixes
iesahin Jul 26, 2024
76a85a4
ws updates
iesahin Jul 26, 2024
bd4688d
update command
iesahin Jul 26, 2024
414351f
fix line items
iesahin Jul 26, 2024
8b4b11f
revert command
iesahin Jul 26, 2024
9d59c6b
fix command
iesahin Jul 26, 2024
0748044
Fixed line errors
iesahin Jul 26, 2024
1752495
added value names
iesahin Jul 26, 2024
4de345b
Added help text for sqlite
iesahin Jul 26, 2024
beeaac8
update help text
iesahin Jul 26, 2024
a467f80
fix sql commands
iesahin Jul 26, 2024
502e8ae
update
iesahin Jul 26, 2024
1fe870f
debug
iesahin Jul 26, 2024
66a4a00
remove comments
iesahin Jul 26, 2024
bfb45fc
rerun
iesahin Jul 26, 2024
17e3dec
update
iesahin Jul 26, 2024
1ac90d2
added people,db
iesahin Jul 26, 2024
29f9329
added people.db
iesahin Jul 26, 2024
d33147a
added people
iesahin Jul 26, 2024
0939cfa
renamed the file
iesahin Jul 26, 2024
55a80bf
remove verbosity
iesahin Jul 26, 2024
369fe32
added compare
iesahin Jul 26, 2024
5dae1d7
fix superficial
iesahin Jul 26, 2024
926b8be
bump version
iesahin Jul 26, 2024
066b856
debug
iesahin Jul 26, 2024
4311bb0
debug
iesahin Jul 28, 2024
df6e2e8
remove watch in select
iesahin Jul 28, 2024
817eb36
handle sqlite type
iesahin Jul 28, 2024
d063ad1
Debug off
iesahin Jul 28, 2024
9fad80e
added bundled sqlite feature
iesahin Jul 28, 2024
0355ebf
renamed the feature and added to main
iesahin Jul 28, 2024
d0f65a9
rename feature
iesahin Jul 28, 2024
8a377a1
fix
iesahin Jul 28, 2024
2e5770d
remove debugs
iesahin Jul 28, 2024
8fe3c8b
fix
iesahin Jul 28, 2024
597a2f9
fix
iesahin Jul 28, 2024
7d39958
default = []
iesahin Jul 28, 2024
dc91b5c
recheck fixes
iesahin Jul 28, 2024
909fa79
add sqlite3 to binaries
iesahin Jul 28, 2024
b4b2bcf
fix ref rest
iesahin Jul 28, 2024
12d74b9
add changes
iesahin Jul 28, 2024
44f2f83
Move reflink behind feature flag
iesahin Jul 28, 2024
2f633a7
fixing feature flags
iesahin Jul 28, 2024
8232d02
add reflink feature to core
iesahin Jul 28, 2024
5f48ad1
remove reflink feature from core
iesahin Jul 28, 2024
84bb318
remove reflink feature from core
iesahin Jul 28, 2024
21e1714
Turn off debug flags
iesahin Jul 28, 2024
29fc013
add compile without default features
iesahin Jul 28, 2024
3d8b1ab
update
iesahin Jul 28, 2024
77fed5c
Added docs dir to lib
iesahin Jul 28, 2024
6b60a8e
Remove workflow_tests
iesahin Jul 28, 2024
9fbd0f6
remove workflow crate refs
iesahin Jul 28, 2024
9f946ac
add test features
iesahin Jul 28, 2024
2b862f2
ws
iesahin Jul 28, 2024
727ddcf
Bump version
iesahin Jul 28, 2024
9cdb635
Merge remote-tracking branch 'origin/main' into develop
iesahin Jul 28, 2024
e719133
fix import errors
iesahin Jul 28, 2024
281f7ed
fix warning
iesahin Jul 28, 2024
2bd4ea7
fix warnings
iesahin Jul 28, 2024
5a70d9d
update
iesahin Jul 28, 2024
8494bdf
fix warnings
iesahin Jul 28, 2024
f6a7510
fix warnings
iesahin Jul 28, 2024
b523cd0
fix some more warnings
iesahin Jul 28, 2024
35e3c6a
turn off wasabi and gcs warnings
iesahin Jul 28, 2024
2232b52
bump version
iesahin Jul 28, 2024
a30cc3d
fix docs errors
iesahin Jul 28, 2024
eb973fb
debug yaml
iesahin Jul 29, 2024
50fdbdf
Add sqlite_bundled feature to Windows build
iesahin Jul 31, 2024
d637f2e
Update pipeline output
iesahin Aug 2, 2024
998dc35
Fix spelling
iesahin Aug 3, 2024
b05e8e7
remove caching from glob_paths, glob_includes and dir_includes
iesahin Aug 3, 2024
fc5b481
remove some traces
iesahin Aug 3, 2024
116b29e
spelling
iesahin Aug 3, 2024
a76215c
tidy up comparisons and add a Diff implementation
iesahin Aug 3, 2024
4f73961
remove some traces
iesahin Aug 3, 2024
8a80345
update superficial and thorough comparisons for glob_items
iesahin Aug 3, 2024
13dba31
Updated docs and changelog
iesahin Aug 3, 2024
2ce46e6
Merge remote-tracking branch 'origin/main' into develop
iesahin Aug 3, 2024
b057fed
Bump version
iesahin Aug 3, 2024
1a35f56
make reflink optional in libn
iesahin Aug 3, 2024
9b1b13d
fix manifest
iesahin Aug 3, 2024
cfa1d49
Fix optional dependencies
iesahin Aug 4, 2024
634a8aa
Added bundled-openssl feature
iesahin Aug 4, 2024
c63f525
Update ref docs showing pipeline outputs
iesahin Aug 4, 2024
f383881
Bump version
iesahin Aug 4, 2024
8d7ad80
Update to use nightly proper
iesahin Aug 4, 2024
dd08d0d
Add publish github action
iesahin Aug 4, 2024
4b5d6a2
Fix publish.yml
iesahin Aug 4, 2024
ac0be74
Use an action for publish crate
iesahin Aug 4, 2024
62d3bf3
fix another fix
iesahin Aug 4, 2024
49d3c68
just use a for loop
iesahin Aug 4, 2024
63d964f
Add keep going when the packages are uploaded
iesahin Aug 4, 2024
0492632
Fix Windows build feature
iesahin Aug 4, 2024
15811a0
Update lib Cargo
iesahin Aug 5, 2024
b966591
bump dependencies
iesahin Aug 5, 2024
c43cdb1
Changelog
iesahin Aug 5, 2024
58f55cf
Fix check_ignore not reading the config
iesahin Aug 6, 2024
d01edab
Bump version
iesahin Aug 6, 2024
66586fd
Bump version
iesahin Aug 6, 2024
52c6870
Bump version
iesahin Aug 6, 2024
eda1658
debug with PEAK_ALLOC
iesahin Aug 6, 2024
b111877
Bump version
iesahin Aug 6, 2024
950c431
ws
iesahin Aug 8, 2024
aae241f
remove GlobSetError
iesahin Aug 8, 2024
70fc0d6
minor changes
iesahin Aug 8, 2024
0d1173b
use fast-glob
iesahin Aug 8, 2024
9def90e
add build_ignore_glob
iesahin Aug 8, 2024
09e349b
remove memory watches
iesahin Aug 8, 2024
1b05248
cleanup
iesahin Aug 8, 2024
e778c16
update to use glob
iesahin Aug 8, 2024
a1ac002
simplify
iesahin Aug 8, 2024
15ed2b9
remove watches and change to fast_glob
iesahin Aug 8, 2024
a0bfe41
remove glob erro
iesahin Aug 8, 2024
f5d5b54
use jwalk and fast-glob
iesahin Aug 8, 2024
8ad1ffb
Remove globset and use fast_glob in ignore
iesahin Aug 12, 2024
1fb26ce
Update SharedIgnoreRules to use RwLock
iesahin Aug 14, 2024
e5461ef
fixed walker_serial
iesahin Aug 14, 2024
49e3da7
fix warnings
iesahin Aug 14, 2024
e995543
fix warning
iesahin Aug 15, 2024
cadebbe
fix watch
iesahin Aug 15, 2024
5fa24cc
fixed file_recheck
iesahin Aug 15, 2024
b0d9ec2
fixed gitignore handler
iesahin Aug 15, 2024
5e2145b
bump version
iesahin Aug 15, 2024
ff8dde8
Merge remote-tracking branch 'origin/main' into develop
iesahin Aug 15, 2024
df81ce0
Fix test_walker_parallel
iesahin Aug 15, 2024
decef74
added symlink check
iesahin Aug 24, 2024
62b7eae
debug
iesahin Aug 24, 2024
5110d00
final /
iesahin Aug 24, 2024
0ac2237
recheck
iesahin Aug 24, 2024
3d8eebd
debug
iesahin Aug 24, 2024
75802ec
update store filtering
iesahin Aug 24, 2024
db4a27d
update sender
iesahin Aug 24, 2024
f6e1c67
list rerun
iesahin Aug 24, 2024
30015f4
Update
iesahin Aug 24, 2024
ed76f8b
update
iesahin Aug 24, 2024
9e1ee9a
add patterns to xvcignore
iesahin Aug 24, 2024
5228f72
check ignore
iesahin Aug 26, 2024
6723326
Update
iesahin Aug 26, 2024
8ab26bc
debug
iesahin Aug 26, 2024
737edd8
debug
iesahin Aug 26, 2024
9082eda
debug
iesahin Aug 26, 2024
7b7669a
fix
iesahin Aug 26, 2024
5fc7e20
fix
iesahin Aug 26, 2024
12a1b14
details
iesahin Aug 26, 2024
85efa22
remove details flag from check ignore
iesahin Aug 26, 2024
e83083a
Added whitelist example
iesahin Aug 26, 2024
2a38eee
fix whitelist example
iesahin Aug 26, 2024
921afbb
Fixed check ignore
iesahin Aug 26, 2024
e8114e2
Update changelog
iesahin Aug 26, 2024
392ddd1
debug
iesahin Aug 27, 2024
b672e3d
Handle directories in store differently
iesahin Aug 27, 2024
4c67cef
modify all store selectors
iesahin Aug 27, 2024
e9f42c2
debug
iesahin Aug 27, 2024
f3ddb36
debug
iesahin Aug 27, 2024
8fc8682
debug
iesahin Aug 27, 2024
d86b8ab
debug
iesahin Aug 27, 2024
6bf1703
debug
iesahin Aug 27, 2024
fbaf049
debug
iesahin Aug 27, 2024
be7cdc5
debug
iesahin Aug 27, 2024
7960fec
debug
iesahin Aug 27, 2024
54be172
fixed?
iesahin Aug 27, 2024
8d4c637
ws
iesahin Aug 27, 2024
fddd0e2
fixed some warnings
iesahin Aug 27, 2024
291a20c
changelog
iesahin Aug 27, 2024
da9e51c
bump version
iesahin Aug 27, 2024
fea9f27
add run-tests to gitignore
iesahin Aug 28, 2024
98cdeb0
debug
iesahin Aug 28, 2024
e004399
Updated example url
iesahin Aug 28, 2024
b6a5dfb
minor
iesahin Aug 28, 2024
e617be6
update test server urls
iesahin Aug 30, 2024
a28b030
Added run-tests script
iesahin Aug 30, 2024
d26def3
test
iesahin Aug 30, 2024
90da278
test
iesahin Aug 30, 2024
3ed4a27
minio
iesahin Aug 30, 2024
80f8d0a
fix urls
iesahin Aug 30, 2024
365c086
test
iesahin Aug 30, 2024
953400e
test
iesahin Aug 30, 2024
28efd73
fix bucket name
iesahin Aug 30, 2024
cfbfe3e
test
iesahin Aug 30, 2024
236f3fa
features
iesahin Aug 30, 2024
ea8544c
fix tests
iesahin Aug 30, 2024
265e35d
fix gitignore traversal
iesahin Aug 30, 2024
070f4e8
test
iesahin Aug 30, 2024
e27d57b
remove some unnecessary tests
iesahin Aug 30, 2024
ca5d534
fix test
iesahin Aug 30, 2024
539b79a
tests
iesahin Aug 30, 2024
5efb633
update
iesahin Aug 30, 2024
9d006b6
changelog
iesahin Aug 30, 2024
6749395
fail fast in tests
iesahin Sep 1, 2024
66e6033
test local
iesahin Sep 1, 2024
b499b18
updated doc tests
iesahin Sep 1, 2024
97d79c2
update test
iesahin Sep 1, 2024
06c88a4
test
iesahin Sep 1, 2024
55add13
config
iesahin Sep 1, 2024
80e7c65
update
iesahin Sep 1, 2024
bca12b6
Update minio tests
iesahin Sep 3, 2024
5dbeb34
fix alias command
iesahin Sep 3, 2024
9131e85
fix
iesahin Sep 4, 2024
c0b8ecf
add another chmod
iesahin Sep 4, 2024
0057570
fix
iesahin Sep 4, 2024
edcd8f1
debug
iesahin Sep 4, 2024
6c29d34
change counting
iesahin Sep 4, 2024
c1b7e05
change release date
iesahin Sep 4, 2024
d09162b
run in shell
iesahin Sep 4, 2024
c3dfc08
Use config file instead of mc alias
iesahin Sep 6, 2024
94d1713
add one-off tests to workflow
iesahin Sep 6, 2024
607c7c0
Update changelog
iesahin Sep 6, 2024
168c3e1
update test logging to trace
iesahin Sep 6, 2024
483cbe7
Added title to Changelog
iesahin Sep 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update test logging to trace
iesahin committed Sep 6, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 168c3e1e1330de72c28c903bea56d4290b298c48
2 changes: 1 addition & 1 deletion lib/tests/test_storage_new_minio.rs
Original file line number Diff line number Diff line change
@@ -76,7 +76,7 @@ fn test_storage_new_minio() -> Result<()> {
};

let x = |cmd: &[&str]| -> Result<String> {
common::run_xvc(Some(&xvc_root), cmd, XvcVerbosity::Warn)
common::run_xvc(Some(&xvc_root), cmd, XvcVerbosity::Trace)
};

let out = x(&[

Unchanged files with check annotations Beta

use std::{
path::Path,
sync::{Arc, Mutex},

Check warning on line 3 in walker/src/walk_parallel.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `Mutex`

Check warning on line 3 in walker/src/walk_parallel.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `Mutex`
};
use crossbeam::queue::SegQueue;
use crossbeam_channel::Sender;
use xvc_logging::{uwr, watch};

Check warning on line 8 in walker/src/walk_parallel.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `uwr`

Check warning on line 8 in walker/src/walk_parallel.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `uwr`
use crate::{
directory_list, update_ignore_rules, IgnoreRules, MatchResult, PathMetadata, Result,

Check warning on line 11 in walker/src/walk_parallel.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `IgnoreRules`

Check warning on line 11 in walker/src/walk_parallel.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `IgnoreRules`
SharedIgnoreRules, WalkOptions, MAX_THREADS_PARALLEL_WALK,
};
pub mod notify;
pub mod pattern;
pub mod sync;
pub mod walk_parallel;

Check warning on line 17 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

missing documentation for a module

Check warning on line 17 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

missing documentation for a module
pub mod walk_serial;
pub use pattern::MatchResult;
pub use ignore_rules::SharedIgnoreRules;
pub use notify::make_watcher;
use std::ffi::OsStr;

Check warning on line 37 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `std::ffi::OsStr`

Check warning on line 37 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `std::ffi::OsStr`
pub use std::hash::Hash;
pub use sync::{PathSync, PathSyncSingleton};
use xvc_logging::warn;
// use glob::{MatchOptions, Pattern, PatternError};
pub use fast_glob::Glob;
use std::{
ffi::OsString,

Check warning on line 50 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `ffi::OsString`

Check warning on line 50 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `ffi::OsString`
fmt::Debug,
fs::{self, Metadata},
path::{Path, PathBuf},
};
use anyhow::{anyhow, Context};

Check warning on line 56 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `Context`

Check warning on line 56 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

unused import: `Context`
static MAX_THREADS_PARALLEL_WALK: usize = 8;
patterns
}
pub fn update_ignore_rules(dir: &Path, ignore_rules: &IgnoreRules) -> Result<()> {

Check warning on line 216 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

missing documentation for a function

Check warning on line 216 in walker/src/lib.rs

GitHub Actions / Test and Coverage (nightly)

missing documentation for a function
if let Some(ref ignore_filename) = ignore_rules.ignore_filename {
let ignore_root = &ignore_rules.root;
let ignore_path = dir.join(ignore_filename);
}
}
pub fn from_global_patterns(

Check warning on line 43 in walker/src/ignore_rules.rs

GitHub Actions / Test and Coverage (nightly)

missing documentation for an associated function

Check warning on line 43 in walker/src/ignore_rules.rs

GitHub Actions / Test and Coverage (nightly)

missing documentation for an associated function
ignore_root: &Path,
ignore_filename: Option<&str>,
given: &str,
},
/// Pattern is from CLI
CommandLine { current_dir: PathBuf },

Check warning on line 75 in walker/src/pattern.rs

GitHub Actions / Test and Coverage (nightly)

missing documentation for a struct field

Check warning on line 75 in walker/src/pattern.rs

GitHub Actions / Test and Coverage (nightly)

missing documentation for a struct field
}
/// Pattern is generic and could be an instance of String, Glob, Regex or any other object.