Class GC
FileRepository
. Instances of
this class are not thread-safe. Don't use the same instance from multiple
threads.
This class started as a copy of DfsGarbageCollector from Shawn O. Pearce
adapted to FileRepositories.-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic class
A class holding statistical data for a FileRepository regarding how many objects are stored as loose or packed objects -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate boolean
Whether gc should do automatic housekeepingprivate boolean
Whether to run gc in a background threadprivate static final String
private static final int
private static final int
private static ExecutorService
private Date
private long
private static final String
private static final String
private Collection<Ref>
the refs which existed during the last call torepack()
.private long
Holds the starting time of the last repack() execution.private static final org.slf4j.Logger
private static final String
private Date
private long
private static final Pattern
private PackConfig
private ProgressMonitor
private static final String
private static final String
private final FileRepository
-
Constructor Summary
ConstructorsConstructorDescriptionGC
(FileRepository repo) Creates a new garbage collector with default values. -
Method Summary
Modifier and TypeMethodDescriptionprivate void
private boolean
canBeSafelyDeleted
(Path path, Instant threshold) private void
private void
private void
private void
private void
deleteOldPacks
(Collection<Pack> oldPacks, Collection<Pack> newPacks) Delete old pack files.private void
Deletes orphansprivate void
private Collection<Pack>
doGc()
private static boolean
private ExecutorService
executor()
gc()
Runs a garbage collector on aFileRepository
.private Collection<Ref>
Returns a collection of all refs and additional refs.private long
private int
private long
private String
Returns information about objects and pack files for a FileRepository.private boolean
isDirectory
(Path p) private static boolean
private static boolean
Return a list of those objects in the index which differ from whats in HEADlistRefLogObjects
(Ref ref, long minTime) private void
loosen
(ObjectDirectoryInserter inserter, ObjectReader reader, Pack pack, HashSet<ObjectId> existing) Loosen objects in a pack file which are not also in the newly-created pack files.private boolean
needGc()
void
packRefs()
Pack ref storage.void
Like "git prune" this method tries to prune all loose objects which are unreferenced.private void
Delete files associated with a single pack file.void
Like "git prune-packed" this method tries to prune all loose objects which can be found in packs.private void
Delete the preserved directory including all pack files withinprivate void
removeOldPack
(PackFile packFile, int deleteOptions) Deletes old pack file, unless 'preserve-oldpacks' is set, in which case it moves the pack file to the preserved directoryprivate void
removeReferenced
(Map<ObjectId, File> id2File, ObjectWalk w) Remove all entries from a map which key is the id of an object referenced by the given ObjectWalkrepack()
Packs all objects which reachable from any of the heads into one pack file.void
setAuto
(boolean auto) Set thegc --auto
option.(package private) void
setBackground
(boolean background) static void
Set the executor for running auto-gc in the background.void
During gc() or prune() each unreferenced, loose object which has been created or modified after or atexpire
will not be pruned.void
setExpireAgeMillis
(long expireAgeMillis) During gc() or prune() each unreferenced, loose object which has been created or modified in the lastexpireAgeMillis
milliseconds will not be pruned.void
setPackConfig
(PackConfig pconfig) Set the PackConfig used when (re-)writing packfiles.void
setPackExpire
(Date packExpire) During gc() or prune() packfiles which are created or modified after or atpackExpire
will not be deleted.void
setPackExpireAgeMillis
(long packExpireAgeMillis) During gc() or prune() packfiles which are created or modified in the lastpackExpireAgeMillis
milliseconds will not be deleted.Set the progress monitor used for garbage collection methods.(package private) boolean
Quickly estimate number of loose objects, SHA1 is distributed evenly so counting objects in one directory (bucket 17) is sufficient(package private) boolean
private Pack
writePack
(Set<? extends ObjectId> want, Set<? extends ObjectId> have, Set<ObjectId> tags, Set<ObjectId> tagTargets, List<ObjectIdSet> excludeObjects)
-
Field Details
-
LOG
private static final org.slf4j.Logger LOG -
PRUNE_EXPIRE_DEFAULT
- See Also:
-
PRUNE_PACK_EXPIRE_DEFAULT
- See Also:
-
PATTERN_LOOSE_OBJECT
-
PACK_EXT
-
BITMAP_EXT
-
INDEX_EXT
-
KEEP_EXT
-
DEFAULT_AUTOPACKLIMIT
private static final int DEFAULT_AUTOPACKLIMIT- See Also:
-
DEFAULT_AUTOLIMIT
private static final int DEFAULT_AUTOLIMIT- See Also:
-
executor
-
repo
-
pm
-
expireAgeMillis
private long expireAgeMillis -
expire
-
packExpireAgeMillis
private long packExpireAgeMillis -
packExpire
-
pconfig
-
lastPackedRefs
the refs which existed during the last call torepack()
. This is needed duringprune(Set)
where we can optimize by looking at the difference between the current refs and the refs which existed during lastrepack()
. -
lastRepackTime
private long lastRepackTimeHolds the starting time of the last repack() execution. This is needed in prune() to inspect only those reflog entries which have been added since last repack(). -
automatic
private boolean automaticWhether gc should do automatic housekeeping -
background
private boolean backgroundWhether to run gc in a background thread
-
-
Constructor Details
-
GC
Creates a new garbage collector with default values. An expirationTime of two weeks andnull
as progress monitor will be used.- Parameters:
repo
- the repo to work on
-
-
Method Details
-
setExecutor
Set the executor for running auto-gc in the background. If no executor is set JGit's own WorkQueue will be used.- Parameters:
e
- the executor to be used for running auto-gc
-
gc
Runs a garbage collector on aFileRepository
. It will- pack loose references into packed-refs
- repack all reachable objects into new pack files and delete the old pack files
- prune all loose objects which are now reachable by packs
setAuto(boolean)
was set totrue
gc
will first check whether any housekeeping is required; if not, it exits without performing any work. IfsetBackground(boolean)
was set totrue
collectGarbage
will start the gc in the background, and then return immediately. In this case, errors will not be reported except in gc.log.- Returns:
- the collection of
Pack
's which are newly created - Throws:
IOException
ParseException
- If the configuration parameter "gc.pruneexpire" couldn't be parsed
-
executor
-
doGc
- Throws:
IOException
ParseException
-
loosen
private void loosen(ObjectDirectoryInserter inserter, ObjectReader reader, Pack pack, HashSet<ObjectId> existing) throws IOException Loosen objects in a pack file which are not also in the newly-created pack files.- Parameters:
inserter
-reader
-pack
-existing
-- Throws:
IOException
-
deleteOldPacks
private void deleteOldPacks(Collection<Pack> oldPacks, Collection<Pack> newPacks) throws ParseException, IOException Delete old pack files. What is 'old' is defined by specifying a set of old pack files and a set of new pack files. Each pack file contained in old pack files but not contained in new pack files will be deleted. If preserveOldPacks is set, keep a copy of the pack file in the preserve directory. If an expirationDate is set then pack files which are younger than the expirationDate will not be deleted nor preserved.If we're not immediately expiring loose objects, loosen any objects in the old pack files which aren't in the new pack files.
- Parameters:
oldPacks
-newPacks
-- Throws:
ParseException
IOException
-
removeOldPack
Deletes old pack file, unless 'preserve-oldpacks' is set, in which case it moves the pack file to the preserved directory- Parameters:
packFile
-deleteOptions
-- Throws:
IOException
-
prunePreserved
private void prunePreserved()Delete the preserved directory including all pack files within -
prunePack
Delete files associated with a single pack file. First try to delete the ".pack" file because on some platforms the ".pack" file may be locked and can't be deleted. In such a case it is better to detect this early and give up on deleting files for this packfile. Otherwise we may delete the ".index" file and when failing to delete the ".pack" file we are left with a ".pack" file without a ".index" file.- Parameters:
packFile
-
-
prunePacked
Like "git prune-packed" this method tries to prune all loose objects which can be found in packs. If certain objects can't be pruned (e.g. because the filesystem delete operation fails) this is silently ignored.- Throws:
IOException
-
prune
Like "git prune" this method tries to prune all loose objects which are unreferenced. If certain objects can't be pruned (e.g. because the filesystem delete operation fails) this is silently ignored.- Parameters:
objectsToKeep
- a set of objects which should explicitly not be pruned- Throws:
IOException
ParseException
- If the configuration parameter "gc.pruneexpire" couldn't be parsed
-
getExpireDate
- Throws:
ParseException
-
getPruneExpireStr
-
getPackExpireDate
- Throws:
ParseException
-
removeReferenced
private void removeReferenced(Map<ObjectId, File> id2File, ObjectWalk w) throws MissingObjectException, IncorrectObjectTypeException, IOExceptionRemove all entries from a map which key is the id of an object referenced by the given ObjectWalk- Parameters:
id2File
-w
-- Throws:
MissingObjectException
IncorrectObjectTypeException
IOException
-
equals
-
packRefs
Pack ref storage. For a RefDirectory database, this packs all non-symbolic, loose refs into packed-refs. For Reftable, all of the data is compacted into a single table.- Throws:
IOException
-
repack
Packs all objects which reachable from any of the heads into one pack file. Additionally all objects which are not reachable from any head but which are reachable from any of the other refs (e.g. tags), special refs (e.g. FETCH_HEAD) or index are packed into a separate pack file. Objects included in pack files which have a .keep file associated are never repacked. All old pack files which existed before are deleted.- Returns:
- a collection of the newly created pack files
- Throws:
IOException
- when during reading of refs, index, packfiles, objects, reflog-entries or during writing to the packfilesIOException
occurs
-
isHead
-
isTag
-
deleteEmptyRefsFolders
- Throws:
IOException
-
canBeSafelyDeleted
-
deleteDir
-
isDirectory
-
delete
-
deleteOrphans
private void deleteOrphans()Deletes orphansA file is considered an orphan if it is either a "bitmap" or an index file, and its corresponding pack file is missing in the list.
-
deleteTempPacksIdx
private void deleteTempPacksIdx() -
listRefLogObjects
- Parameters:
ref
- the ref which log should be inspectedminTime
- only reflog entries not older then this time are processed- Returns:
- the
ObjectId
s contained in the reflog - Throws:
IOException
-
getAllRefs
Returns a collection of all refs and additional refs. Additional refs which don't start with "refs/" are not returned because they should not save objects from being garbage collected. Examples for such references are ORIG_HEAD, MERGE_HEAD, FETCH_HEAD and CHERRY_PICK_HEAD.- Returns:
- a collection of refs pointing to live objects.
- Throws:
IOException
-
listNonHEADIndexObjects
Return a list of those objects in the index which differ from whats in HEAD- Returns:
- a set of ObjectIds of changed objects in the index
- Throws:
IOException
CorruptObjectException
NoWorkTreeException
-
writePack
private Pack writePack(@NonNull Set<? extends ObjectId> want, @NonNull Set<? extends ObjectId> have, @NonNull Set<ObjectId> tags, Set<ObjectId> tagTargets, List<ObjectIdSet> excludeObjects) throws IOException - Throws:
IOException
-
checkCancelled
- Throws:
CancelledException
-
getStatistics
Returns information about objects and pack files for a FileRepository.- Returns:
- information about objects and pack files for a FileRepository
- Throws:
IOException
-
setProgressMonitor
Set the progress monitor used for garbage collection methods.- Parameters:
pm
- aProgressMonitor
object.- Returns:
- this
-
setExpireAgeMillis
public void setExpireAgeMillis(long expireAgeMillis) During gc() or prune() each unreferenced, loose object which has been created or modified in the lastexpireAgeMillis
milliseconds will not be pruned. Only older objects may be pruned. If set to 0 then every object is a candidate for pruning.- Parameters:
expireAgeMillis
- minimal age of objects to be pruned in milliseconds.
-
setPackExpireAgeMillis
public void setPackExpireAgeMillis(long packExpireAgeMillis) During gc() or prune() packfiles which are created or modified in the lastpackExpireAgeMillis
milliseconds will not be deleted. Only older packfiles may be deleted. If set to 0 then every packfile is a candidate for deletion.- Parameters:
packExpireAgeMillis
- minimal age of packfiles to be deleted in milliseconds.
-
setPackConfig
Set the PackConfig used when (re-)writing packfiles. This allows to influence how packs are written and to implement something similar to "git gc --aggressive"- Parameters:
pconfig
- thePackConfig
used when writing packs
-
setExpire
During gc() or prune() each unreferenced, loose object which has been created or modified after or atexpire
will not be pruned. Only older objects may be pruned. If set to null then every object is a candidate for pruning.- Parameters:
expire
- instant in time which defines object expiration objects with modification time before this instant are expired objects with modification time newer or equal to this instant are not expired
-
setPackExpire
During gc() or prune() packfiles which are created or modified after or atpackExpire
will not be deleted. Only older packfiles may be deleted. If set to null then every packfile is a candidate for deletion.- Parameters:
packExpire
- instant in time which defines packfile expiration
-
setAuto
public void setAuto(boolean auto) Set thegc --auto
option. With this option, gc checks whether any housekeeping is required; if not, it exits without performing any work. Some JGit commands rungc --auto
after performing operations that could create many loose objects.Housekeeping is required if there are too many loose objects or too many packs in the repository. If the number of loose objects exceeds the value of the gc.auto option JGit GC consolidates all existing packs into a single pack (equivalent to
-A
option), whereas git-core would combine all loose objects into a single pack usingrepack -d -l
. Setting the value ofgc.auto
to 0 disables automatic packing of loose objects.If the number of packs exceeds the value of
gc.autoPackLimit
, then existing packs (except those marked with a .keep file) are consolidated into a single pack by using the-A
option of repack. Settinggc.autoPackLimit
to 0 disables automatic consolidation of packs.Like git the following jgit commands run auto gc:
- fetch
- merge
- rebase
- receive-pack
receive.autogc = false
- Parameters:
auto
- defines whether gc should do automatic housekeeping
-
setBackground
void setBackground(boolean background) - Parameters:
background
- whether to run the gc in a background thread.
-
needGc
private boolean needGc() -
addRepackAllOption
private void addRepackAllOption() -
tooManyPacks
boolean tooManyPacks()- Returns:
true
if number of packs > gc.autopacklimit (default 50)
-
tooManyLooseObjects
boolean tooManyLooseObjects()Quickly estimate number of loose objects, SHA1 is distributed evenly so counting objects in one directory (bucket 17) is sufficient- Returns:
true
if number of loose objects > gc.auto (default 6700)
-
getLooseObjectLimit
private int getLooseObjectLimit()
-