Class UploadPack

java.lang.Object
org.eclipse.jgit.transport.UploadPack

public class UploadPack extends Object
Implements the server side of a fetch connection, transmitting objects.
  • Field Details

    • db

      private final Repository db
      Database we read the objects from.
    • walk

      private final RevWalk walk
      Revision traversal support over db.
    • packConfig

      private PackConfig packConfig
      Configuration to pass into the PackWriter.
    • transferConfig

      private TransferConfig transferConfig
      Configuration for various transfer options.
    • timeout

      private int timeout
      Timeout in seconds to wait for client interaction.
    • biDirectionalPipe

      private boolean biDirectionalPipe
      Is the client connection a bi-directional socket or pipe?

      If true, this class assumes it can perform multiple read and write cycles with the client over the input and output streams. This matches the functionality available with a standard TCP/IP connection, or a local operating system or in-memory pipe.

      If false, this class runs in a read everything then output results mode, making it suitable for single round-trip systems RPCs such as HTTP.

    • timer

      private InterruptTimer timer
      Timer to manage timeout.
    • clientRequestedV2

      private boolean clientRequestedV2
      Whether the client requested to use protocol V2 through a side channel (such as the Git-Protocol HTTP header).
    • rawIn

      private InputStream rawIn
    • rawOut

    • pckIn

      private PacketLineIn pckIn
    • msgOut

      private OutputStream msgOut
    • errOut

      private UploadPack.ErrorWriter errOut
    • refs

      private Map<String,Ref> refs
      Refs eligible for advertising to the client, set using setAdvertisedRefs(java.util.Map<java.lang.String, org.eclipse.jgit.lib.Ref>).
    • protocolV2Hook

      private ProtocolV2Hook protocolV2Hook
      Hook used while processing Git protocol v2 requests.
    • advertiseRefsHook

      private AdvertiseRefsHook advertiseRefsHook
      Hook used while advertising the refs to the client.
    • advertiseRefsHookCalled

      private boolean advertiseRefsHookCalled
      Whether the advertiseRefsHook has been invoked.
    • refFilter

      private RefFilter refFilter
      Filter used while advertising the refs to the client.
    • preUploadHook

      private PreUploadHook preUploadHook
      Hook handling the various upload phases.
    • postUploadHook

      private PostUploadHook postUploadHook
      Hook for taking post upload actions.
    • userAgent

      String userAgent
      Caller user agent
    • wantIds

      private Set<ObjectId> wantIds
      Raw ObjectIds the client has asked for, before validating them.
    • wantAll

      private final Set<RevObject> wantAll
      Objects the client wants to obtain.
    • commonBase

      private final Set<RevObject> commonBase
      Objects on both sides, these don't have to be sent.
    • oldestTime

      private int oldestTime
      Commit time of the oldest common commit, in seconds.
    • okToGiveUp

      private Boolean okToGiveUp
      null if commonBase should be examined again.
    • sentReady

      private boolean sentReady
    • advertised

      private Set<ObjectId> advertised
      Objects we sent in our advertisement list.
    • WANT

      private final RevFlag WANT
      Marked on objects the client has asked us to give them.
    • PEER_HAS

      private final RevFlag PEER_HAS
      Marked on objects both we and the client have.
    • COMMON

      private final RevFlag COMMON
      Marked on objects in commonBase.
    • SATISFIED

      private final RevFlag SATISFIED
      Objects where we found a path from the want list to a common base.
    • SAVE

      private final RevFlagSet SAVE
    • requestValidator

      private UploadPack.RequestValidator requestValidator
    • multiAck

    • noDone

      private boolean noDone
    • statistics

      private PackStatistics statistics
    • currentRequest

      private FetchRequest currentRequest
      Request this instance is handling. We need to keep a reference to it for pre upload hooks. They receive a reference this instance and invoke methods like getDepth() to get information about the request.
    • cachedPackUriProvider

      private CachedPackUriProvider cachedPackUriProvider
  • Constructor Details

    • UploadPack

      public UploadPack(Repository copyFrom)
      Create a new pack upload for an open repository.
      Parameters:
      copyFrom - the source repository.
  • Method Details

    • getRepository

      public final Repository getRepository()
      Get the repository this upload is reading from.
      Returns:
      the repository this upload is reading from.
    • getRevWalk

      public final RevWalk getRevWalk()
      Get the RevWalk instance used by this connection.
      Returns:
      the RevWalk instance used by this connection.
    • getAdvertisedRefs

      public final Map<String,Ref> getAdvertisedRefs()
      Get refs which were advertised to the client.
      Returns:
      all refs which were advertised to the client. Only valid during the negotiation phase. Will return null if setAdvertisedRefs(Map) has not been called yet or if #sendPack() has been called.
    • setAdvertisedRefs

      public void setAdvertisedRefs(@Nullable Map<String,Ref> allRefs)
      Set the refs advertised by this UploadPack.

      Intended to be called from a PreUploadHook.

      Parameters:
      allRefs - explicit set of references to claim as advertised by this UploadPack instance. This overrides any references that may exist in the source repository. The map is passed to the configured getRefFilter(). If null, assumes all refs were advertised.
    • getTimeout

      public int getTimeout()
      Get timeout (in seconds) before aborting an IO operation.
      Returns:
      timeout (in seconds) before aborting an IO operation.
    • setTimeout

      public void setTimeout(int seconds)
      Set the timeout before willing to abort an IO call.
      Parameters:
      seconds - number of seconds to wait (with no data transfer occurring) before aborting an IO read or write operation with the connected client.
    • isBiDirectionalPipe

      public boolean isBiDirectionalPipe()
      Whether this class expects a bi-directional pipe opened between the client and itself.
      Returns:
      true if this class expects a bi-directional pipe opened between the client and itself. The default is true.
    • setBiDirectionalPipe

      public void setBiDirectionalPipe(boolean twoWay)
      Set whether this class will assume the socket is a fully bidirectional pipe between the two peers
      Parameters:
      twoWay - if true, this class will assume the socket is a fully bidirectional pipe between the two peers and takes advantage of that by first transmitting the known refs, then waiting to read commands. If false, this class assumes it must read the commands before writing output and does not perform the initial advertising.
    • getRequestPolicy

      public UploadPack.RequestPolicy getRequestPolicy()
      Get policy used by the service to validate client requests
      Returns:
      policy used by the service to validate client requests, or null for a custom request validator.
    • setRequestPolicy

      public void setRequestPolicy(UploadPack.RequestPolicy policy)
      Set the policy used to enforce validation of a client's want list.
      Parameters:
      policy - the policy used to enforce validation of a client's want list. By default the policy is UploadPack.RequestPolicy.ADVERTISED, which is the Git default requiring clients to only ask for an object that a reference directly points to. This may be relaxed to UploadPack.RequestPolicy.REACHABLE_COMMIT or UploadPack.RequestPolicy.REACHABLE_COMMIT_TIP when callers have setBiDirectionalPipe(boolean) set to false. Overrides any policy specified in a TransferConfig.
    • setRequestValidator

      public void setRequestValidator(@Nullable UploadPack.RequestValidator validator)
      Set custom validator for client want list.
      Parameters:
      validator - custom validator for client want list.
      Since:
      3.1
    • getAdvertiseRefsHook

      public AdvertiseRefsHook getAdvertiseRefsHook()
      Get the hook used while advertising the refs to the client.
      Returns:
      the hook used while advertising the refs to the client.
    • getRefFilter

      public RefFilter getRefFilter()
      Get the filter used while advertising the refs to the client.
      Returns:
      the filter used while advertising the refs to the client.
    • setAdvertiseRefsHook

      public void setAdvertiseRefsHook(@Nullable AdvertiseRefsHook advertiseRefsHook)
      Set the hook used while advertising the refs to the client.

      If the AdvertiseRefsHook chooses to call setAdvertisedRefs(Map), only refs set by this hook and selected by the RefFilter will be shown to the client.

      Parameters:
      advertiseRefsHook - the hook; may be null to show all refs.
    • setProtocolV2Hook

      public void setProtocolV2Hook(@Nullable ProtocolV2Hook hook)
      Set the protocol V2 hook.
      Parameters:
      hook - the hook; if null no special actions are taken.
      Since:
      5.1
    • getProtocolV2Hook

      public ProtocolV2Hook getProtocolV2Hook()
      Get the currently installed protocol v2 hook.
      Returns:
      the hook or a default implementation if none installed.
      Since:
      5.5
    • setRefFilter

      public void setRefFilter(@Nullable RefFilter refFilter)
      Set the filter used while advertising the refs to the client.

      Only refs allowed by this filter will be sent to the client. The filter is run against the refs specified by the AdvertiseRefsHook (if applicable). If null or not set, uses the filter implied by the TransferConfig.

      Parameters:
      refFilter - the filter; may be null to show all refs.
    • getPreUploadHook

      public PreUploadHook getPreUploadHook()
      Get the configured pre upload hook.
      Returns:
      the configured pre upload hook.
    • setPreUploadHook

      public void setPreUploadHook(@Nullable PreUploadHook hook)
      Set the hook that controls how this instance will behave.
      Parameters:
      hook - the hook; if null no special actions are taken.
    • getPostUploadHook

      public PostUploadHook getPostUploadHook()
      Get the configured post upload hook.
      Returns:
      the configured post upload hook.
      Since:
      4.1
    • setPostUploadHook

      public void setPostUploadHook(@Nullable PostUploadHook hook)
      Set the hook for post upload actions (logging, repacking).
      Parameters:
      hook - the hook; if null no special actions are taken.
      Since:
      4.1
    • setPackConfig

      public void setPackConfig(@Nullable PackConfig pc)
      Set the configuration used by the pack generator.
      Parameters:
      pc - configuration controlling packing parameters. If null the source repository's settings will be used.
    • setTransferConfig

      public void setTransferConfig(@Nullable TransferConfig tc)
      Set configuration controlling transfer options.
      Parameters:
      tc - configuration controlling transfer options. If null the source repository's settings will be used.
      Since:
      3.1
    • isSideBand

      public boolean isSideBand() throws RequestNotYetReadException
      Check whether the client expects a side-band stream.
      Returns:
      true if the client has advertised a side-band capability, false otherwise.
      Throws:
      RequestNotYetReadException - if the client's request has not yet been read from the wire, so we do not know if they expect side-band. Note that the client may have already written the request, it just has not been read.
    • setExtraParameters

      public void setExtraParameters(Collection<String> params)
      Set the Extra Parameters provided by the client.

      These are parameters passed by the client through a side channel such as the Git-Protocol HTTP header, to allow a client to request a newer response format while remaining compatible with older servers that do not understand different request formats.

      Parameters:
      params - parameters supplied by the client, split at colons or NUL bytes.
      Since:
      5.0
    • setCachedPackUriProvider

      public void setCachedPackUriProvider(@Nullable CachedPackUriProvider p)
      Parameters:
      p - provider of URIs corresponding to cached packs (to support the packfile URIs feature)
      Since:
      5.5
    • useProtocolV2

      private boolean useProtocolV2()
    • upload

      public void upload(InputStream input, OutputStream output, @Nullable OutputStream messages) throws IOException
      Execute the upload task on the socket.

      Same as uploadWithExceptionPropagation(java.io.InputStream, java.io.OutputStream, java.io.OutputStream) except that the thrown exceptions are handled in the method, and the error messages are sent to the clients.

      Call this method if the caller does not have an error handling mechanism. Call uploadWithExceptionPropagation(java.io.InputStream, java.io.OutputStream, java.io.OutputStream) if the caller wants to have its own error handling mechanism.

      Parameters:
      input -
      output -
      messages -
      Throws:
      IOException
    • uploadWithExceptionPropagation

      public void uploadWithExceptionPropagation(InputStream input, OutputStream output, @Nullable OutputStream messages) throws ServiceMayNotContinueException, IOException
      Execute the upload task on the socket.

      If the client passed extra parameters (e.g., "version=2") through a side channel, the caller must call setExtraParameters first to supply them.

      Parameters:
      input - raw input to read client commands from. Caller must ensure the input is buffered, otherwise read performance may suffer.
      output - response back to the Git network client, to write the pack data onto. Caller must ensure the output is buffered, otherwise write performance may suffer.
      messages - secondary "notice" channel to send additional messages out through. When run over SSH this should be tied back to the standard error channel of the command execution. For most other network connections this should be null.
      Throws:
      ServiceMayNotContinueException - thrown if one of the hooks throws this.
      IOException - thrown if the server or the client I/O fails, or there's an internal server error.
      Since:
      5.6
    • getStatistics

      public PackStatistics getStatistics()
      Get the PackWriter's statistics if a pack was sent to the client.
      Returns:
      statistics about pack output, if a pack was sent. Null if no pack was sent, such as during the negotiation phase of a smart HTTP connection, or if the client was already up-to-date.
      Since:
      4.1
    • getAllRefs

      private Map<String,Ref> getAllRefs()
      Extract the full list of refs from the ref-db.
      Returns:
      Map of all refname/ref
    • getAdvertisedOrDefaultRefs

      private Map<String,Ref> getAdvertisedOrDefaultRefs() throws IOException
      Throws:
      IOException
    • getFilteredRefs

      private Map<String,Ref> getFilteredRefs(Collection<String> refPrefixes) throws IOException
      Throws:
      IOException
    • mapRefs

      @NonNull private static Map<String,Ref> mapRefs(Map<String,Ref> refs, List<String> names)
      Returns the specified references.

      This produces an immutable map containing whatever subset of the refs named by the caller are present in the supplied refs map.

      Parameters:
      refs - Map to search for refs to return.
      names - which refs to search for in refs.
      Returns:
      the requested Refs, omitting any that are null or missing.
    • exactRefs

      @NonNull private Map<String,Ref> exactRefs(List<String> names) throws IOException
      Read refs on behalf of the client.

      This checks whether the refs are present in the ref advertisement since otherwise the client might not be supposed to be able to read them.

      Parameters:
      names - unabbreviated names of references.
      Returns:
      the requested Refs, omitting any that are not visible or do not exist.
      Throws:
      IOException - on failure to read a ref or check it for visibility.
    • findRef

      @Nullable private Ref findRef(String name) throws IOException
      Find a ref in the usual search path on behalf of the client.

      This checks that the ref is present in the ref advertisement since otherwise the client might not be supposed to be able to read it.

      Parameters:
      name - short name of the ref to find, e.g. "master" to find "refs/heads/master".
      Returns:
      the requested Ref, or null if it is not visible or does not exist.
      Throws:
      IOException - on failure to read the ref or check it for visibility.
    • service

      private void service(PacketLineOut pckOut) throws IOException
      Throws:
      IOException
    • lsRefsV2

      private void lsRefsV2(PacketLineOut pckOut) throws IOException
      Throws:
      IOException
    • wantedRefs

      private Map<String,ObjectId> wantedRefs(FetchV2Request req) throws IOException
      Throws:
      IOException
    • fetchV2

      private void fetchV2(PacketLineOut pckOut) throws IOException
      Throws:
      IOException
    • objectInfo

      private void objectInfo(PacketLineOut pckOut) throws IOException
      Throws:
      IOException
    • serveOneCommandV2

      private boolean serveOneCommandV2(PacketLineOut pckOut) throws IOException
      Throws:
      IOException
    • getV2CapabilityAdvertisement

      private List<String> getV2CapabilityAdvertisement()
    • serviceV2

      private void serviceV2(PacketLineOut pckOut) throws IOException
      Throws:
      IOException
    • refIdSet

      private static Set<ObjectId> refIdSet(Collection<Ref> refs)
    • computeShallowsAndUnshallows

      private void computeShallowsAndUnshallows(FetchRequest req, UploadPack.IOConsumer<ObjectId> shallowFunc, UploadPack.IOConsumer<ObjectId> unshallowFunc, List<ObjectId> deepenNots) throws IOException
      Throws:
      IOException
    • verifyClientShallow

      private void verifyClientShallow(Set<ObjectId> shallowCommits) throws IOException, PackProtocolException
      Throws:
      IOException
      PackProtocolException
    • sendAdvertisedRefs

      public void sendAdvertisedRefs(RefAdvertiser adv) throws IOException, ServiceMayNotContinueException
      Generate an advertisement of available refs and capabilities.
      Parameters:
      adv - the advertisement formatter.
      Throws:
      IOException - the formatter failed to write an advertisement.
      ServiceMayNotContinueException - the hook denied advertisement.
    • sendAdvertisedRefs

      public void sendAdvertisedRefs(RefAdvertiser adv, @Nullable String serviceName) throws IOException, ServiceMayNotContinueException
      Generate an advertisement of available refs and capabilities.
      Parameters:
      adv - the advertisement formatter.
      serviceName - if not null, also output "# service=serviceName" followed by a flush packet before the advertisement. This is required in v0 of the HTTP protocol, described in Git's Documentation/technical/http-protocol.txt.
      Throws:
      IOException - the formatter failed to write an advertisement.
      ServiceMayNotContinueException - the hook denied advertisement.
      Since:
      5.0
    • sendMessage

      public void sendMessage(String what)
      Send a message to the client, if it supports receiving them.

      If the client doesn't support receiving messages, the message will be discarded, with no other indication to the caller or to the client.

      Parameters:
      what - string describing the problem identified by the hook. The string must not end with an LF, and must not contain an LF.
      Since:
      3.1
    • getMessageOutputStream

      public OutputStream getMessageOutputStream()
      Get an underlying stream for sending messages to the client
      Returns:
      an underlying stream for sending messages to the client, or null.
      Since:
      3.1
    • getDepth

      public int getDepth()
      Returns the clone/fetch depth. Valid only after calling recvWants(). A depth of 1 means return only the wants.
      Returns:
      the depth requested by the client, or 0 if unbounded.
      Since:
      4.0
    • getFilterBlobLimit

      @Deprecated public final long getFilterBlobLimit()
      Deprecated.
      Use getFilterSpec() instead
      Deprecated synonym for getFilterSpec().getBlobLimit().
      Returns:
      filter blob limit requested by the client, or -1 if no limit
      Since:
      5.3
    • getFilterSpec

      public final FilterSpec getFilterSpec()
      Returns the filter spec for the current request. Valid only after calling recvWants(). This may be a no-op filter spec, but it won't be null.
      Returns:
      filter requested by the client
      Since:
      5.4
    • getPeerUserAgent

      public String getPeerUserAgent()
      Get the user agent of the client.

      If the client is new enough to use agent= capability that value will be returned. Older HTTP clients may also supply their version using the HTTP User-Agent header. The capability overrides the HTTP header if both are available.

      When an HTTP request has been received this method returns the HTTP User-Agent header value until capabilities have been parsed.

      Returns:
      user agent supplied by the client. Available only if the client is new enough to advertise its user agent.
      Since:
      4.0
    • negotiate

      private boolean negotiate(FetchRequest req, PackStatistics.Accumulator accumulator, PacketLineOut pckOut) throws IOException
      Throws:
      IOException
    • processHaveLines

      private ObjectId processHaveLines(List<ObjectId> peerHas, ObjectId last, PacketLineOut out, PackStatistics.Accumulator accumulator, UploadPack.Option option) throws IOException
      Throws:
      IOException
    • shouldGiveUp

      private boolean shouldGiveUp(List<ObjectId> peerHas, PacketLineOut out, int missCnt) throws IOException
      Throws:
      IOException
    • parseWants

      private void parseWants(PackStatistics.Accumulator accumulator) throws IOException
      Throws:
      IOException
    • want

      private void want(RevObject obj)
    • checkNotAdvertisedWants

      private static void checkNotAdvertisedWants(UploadPack up, List<ObjectId> notAdvertisedWants, Collection<Ref> visibleRefs) throws IOException
      Throws:
      IOException
    • importantRefsFirst

      static Stream<Ref> importantRefsFirst(Collection<Ref> visibleRefs)
    • refToObjectId

      private static ObjectId refToObjectId(Ref ref)
    • objectIdToRevCommit

      @Nullable private static RevCommit objectIdToRevCommit(RevWalk walk, ObjectId objectId)
      Translate an object id to a RevCommit.
      Parameters:
      walk - walk on the relevant object storae
      objectId - Object Id
      Returns:
      RevCommit instance or null if the object is missing
    • objectIdToRevObject

      @Nullable private static RevObject objectIdToRevObject(RevWalk walk, ObjectId objectId)
      Translate an object id to a RevObject.
      Parameters:
      walk - walk on the relevant object storage
      objectId - Object Id
      Returns:
      RevObject instance or null if the object is missing
    • objectIdsToRevObjects

      private static List<RevObject> objectIdsToRevObjects(RevWalk walk, Iterable<ObjectId> objectIds) throws MissingObjectException, IOException
      Throws:
      MissingObjectException
      IOException
    • addCommonBase

      private void addCommonBase(RevObject o)
    • okToGiveUp

      private boolean okToGiveUp() throws PackProtocolException
      Throws:
      PackProtocolException
    • okToGiveUpImp

      private boolean okToGiveUpImp() throws PackProtocolException
      Throws:
      PackProtocolException
    • wantSatisfied

      private boolean wantSatisfied(RevObject want) throws IOException
      Throws:
      IOException
    • sendPack

      private void sendPack(PackStatistics.Accumulator accumulator, FetchRequest req, @Nullable Collection<Ref> allTags, List<ObjectId> unshallowCommits, List<ObjectId> deepenNots, PacketLineOut pckOut) throws IOException
      Send the requested objects to the client.
      Parameters:
      accumulator - where to write statistics about the content of the pack.
      req - request in process
      allTags - refs to search for annotated tags to include in the pack if the #OPTION_INCLUDE_TAG capability was requested.
      unshallowCommits - shallow commits on the client that are now becoming unshallow
      deepenNots - objects that the client specified using --shallow-exclude
      pckOut - output writer
      Throws:
      IOException - if an error occurred while generating or writing the pack.
    • sendPack

      private void sendPack(ProgressMonitor pm, PacketLineOut pckOut, OutputStream packOut, FetchRequest req, PackStatistics.Accumulator accumulator, @Nullable Collection<Ref> allTags, List<ObjectId> unshallowCommits, List<ObjectId> deepenNots) throws IOException
      Send the requested objects to the client.
      Parameters:
      pm - progress monitor
      pckOut - PacketLineOut that shares the output with packOut
      packOut - packfile output
      req - request being processed
      accumulator - where to write statistics about the content of the pack.
      allTags - refs to search for annotated tags to include in the pack if the #OPTION_INCLUDE_TAG capability was requested.
      unshallowCommits - shallow commits on the client that are now becoming unshallow
      deepenNots - objects that the client specified using --shallow-exclude
      Throws:
      IOException - if an error occurred while generating or writing the pack.
    • findSymrefs

      private static void findSymrefs(RefAdvertiser adv, Map<String,Ref> refs)
    • addTagChain

      private void addTagChain(RevObject o, PackWriter pw) throws IOException
      Throws:
      IOException