Tech-invite3GPPspaceIETFspace
959493929190898887868584838281807978777675747372717069686766656463626160595857565554535251504948474645444342414039383736353433323130292827262524232221201918171615141312111009080706050403020100
in Index   Prev   Next

RFC 3986

Uniform Resource Identifier (URI): Generic Syntax

Pages: 61
Internet Standard: 66
Errata
Obsoletes:  273223961808
Updates:  1738
Updated by:  687473208820
Part 1 of 3 – Pages 1 to 16
None   None   Next

Top   ToC   RFC3986 - Page 1
Network Working Group                                     T. Berners-Lee
Request for Comments: 3986                                       W3C/MIT
STD: 66                                                      R. Fielding
Updates: 1738                                               Day Software
Obsoletes: 2732, 2396, 1808                                  L. Masinter
Category: Standards Track                                  Adobe Systems
                                                            January 2005


           Uniform Resource Identifier (URI): Generic Syntax

Status of This Memo

   This document specifies an Internet standards track protocol for the
   Internet community, and requests discussion and suggestions for
   improvements.  Please refer to the current edition of the "Internet
   Official Protocol Standards" (STD 1) for the standardization state
   and status of this protocol.  Distribution of this memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2005).

Abstract

A Uniform Resource Identifier (URI) is a compact sequence of characters that identifies an abstract or physical resource. This specification defines the generic URI syntax and a process for resolving URI references that might be in relative form, along with guidelines and security considerations for the use of URIs on the Internet. The URI syntax defines a grammar that is a superset of all valid URIs, allowing an implementation to parse the common components of a URI reference without knowing the scheme-specific requirements of every possible identifier. This specification does not define a generative grammar for URIs; that task is performed by the individual specifications of each URI scheme.
Top   ToC   RFC3986 - Page 2

Table of Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1. Overview of URIs . . . . . . . . . . . . . . . . . . . . 4 1.1.1. Generic Syntax . . . . . . . . . . . . . . . . . 6 1.1.2. Examples . . . . . . . . . . . . . . . . . . . . 7 1.1.3. URI, URL, and URN . . . . . . . . . . . . . . . 7 1.2. Design Considerations . . . . . . . . . . . . . . . . . 8 1.2.1. Transcription . . . . . . . . . . . . . . . . . 8 1.2.2. Separating Identification from Interaction . . . 9 1.2.3. Hierarchical Identifiers . . . . . . . . . . . . 10 1.3. Syntax Notation . . . . . . . . . . . . . . . . . . . . 11 2. Characters . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1. Percent-Encoding . . . . . . . . . . . . . . . . . . . . 12 2.2. Reserved Characters . . . . . . . . . . . . . . . . . . 12 2.3. Unreserved Characters . . . . . . . . . . . . . . . . . 13 2.4. When to Encode or Decode . . . . . . . . . . . . . . . . 14 2.5. Identifying Data . . . . . . . . . . . . . . . . . . . . 14 3. Syntax Components . . . . . . . . . . . . . . . . . . . . . . 16 3.1. Scheme . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.2. Authority . . . . . . . . . . . . . . . . . . . . . . . 17 3.2.1. User Information . . . . . . . . . . . . . . . . 18 3.2.2. Host . . . . . . . . . . . . . . . . . . . . . . 18 3.2.3. Port . . . . . . . . . . . . . . . . . . . . . . 22 3.3. Path . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.4. Query . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.5. Fragment . . . . . . . . . . . . . . . . . . . . . . . . 24 4. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.1. URI Reference . . . . . . . . . . . . . . . . . . . . . 25 4.2. Relative Reference . . . . . . . . . . . . . . . . . . . 26 4.3. Absolute URI . . . . . . . . . . . . . . . . . . . . . . 27 4.4. Same-Document Reference . . . . . . . . . . . . . . . . 27 4.5. Suffix Reference . . . . . . . . . . . . . . . . . . . . 27 5. Reference Resolution . . . . . . . . . . . . . . . . . . . . . 28 5.1. Establishing a Base URI . . . . . . . . . . . . . . . . 28 5.1.1. Base URI Embedded in Content . . . . . . . . . . 29 5.1.2. Base URI from the Encapsulating Entity . . . . . 29 5.1.3. Base URI from the Retrieval URI . . . . . . . . 30 5.1.4. Default Base URI . . . . . . . . . . . . . . . . 30 5.2. Relative Resolution . . . . . . . . . . . . . . . . . . 30 5.2.1. Pre-parse the Base URI . . . . . . . . . . . . . 31 5.2.2. Transform References . . . . . . . . . . . . . . 31 5.2.3. Merge Paths . . . . . . . . . . . . . . . . . . 32 5.2.4. Remove Dot Segments . . . . . . . . . . . . . . 33 5.3. Component Recomposition . . . . . . . . . . . . . . . . 35 5.4. Reference Resolution Examples . . . . . . . . . . . . . 35 5.4.1. Normal Examples . . . . . . . . . . . . . . . . 36 5.4.2. Abnormal Examples . . . . . . . . . . . . . . . 36
Top   ToC   RFC3986 - Page 3
   6.  Normalization and Comparison . . . . . . . . . . . . . . . . . 38
       6.1.  Equivalence  . . . . . . . . . . . . . . . . . . . . . . 38
       6.2.  Comparison Ladder  . . . . . . . . . . . . . . . . . . . 39
             6.2.1.  Simple String Comparison . . . . . . . . . . . . 39
             6.2.2.  Syntax-Based Normalization . . . . . . . . . . . 40
             6.2.3.  Scheme-Based Normalization . . . . . . . . . . . 41
             6.2.4.  Protocol-Based Normalization . . . . . . . . . . 42
   7.  Security Considerations  . . . . . . . . . . . . . . . . . . . 43
       7.1.  Reliability and Consistency  . . . . . . . . . . . . . . 43
       7.2.  Malicious Construction . . . . . . . . . . . . . . . . . 43
       7.3.  Back-End Transcoding . . . . . . . . . . . . . . . . . . 44
       7.4.  Rare IP Address Formats  . . . . . . . . . . . . . . . . 45
       7.5.  Sensitive Information  . . . . . . . . . . . . . . . . . 45
       7.6.  Semantic Attacks . . . . . . . . . . . . . . . . . . . . 45
   8.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 46
   9.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 46
   10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 46
       10.1. Normative References . . . . . . . . . . . . . . . . . . 46
       10.2. Informative References . . . . . . . . . . . . . . . . . 47
   A.  Collected ABNF for URI . . . . . . . . . . . . . . . . . . . . 49
   B.  Parsing a URI Reference with a Regular Expression  . . . . . . 50
   C.  Delimiting a URI in Context  . . . . . . . . . . . . . . . . . 51
   D.  Changes from RFC 2396  . . . . . . . . . . . . . . . . . . . . 53
       D.1.  Additions  . . . . . . . . . . . . . . . . . . . . . . . 53
       D.2.  Modifications  . . . . . . . . . . . . . . . . . . . . . 53
   Index  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 60
   Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 61
Top   ToC   RFC3986 - Page 4

1. Introduction

A Uniform Resource Identifier (URI) provides a simple and extensible means for identifying a resource. This specification of URI syntax and semantics is derived from concepts introduced by the World Wide Web global information initiative, whose use of these identifiers dates from 1990 and is described in "Universal Resource Identifiers in WWW" [RFC1630]. The syntax is designed to meet the recommendations laid out in "Functional Recommendations for Internet Resource Locators" [RFC1736] and "Functional Requirements for Uniform Resource Names" [RFC1737]. This document obsoletes [RFC2396], which merged "Uniform Resource Locators" [RFC1738] and "Relative Uniform Resource Locators" [RFC1808] in order to define a single, generic syntax for all URIs. It obsoletes [RFC2732], which introduced syntax for an IPv6 address. It excludes portions of RFC 1738 that defined the specific syntax of individual URI schemes; those portions will be updated as separate documents. The process for registration of new URI schemes is defined separately by [BCP35]. Advice for designers of new URI schemes can be found in [RFC2718]. All significant changes from RFC 2396 are noted in Appendix D. This specification uses the terms "character" and "coded character set" in accordance with the definitions provided in [BCP19], and "character encoding" in place of what [BCP19] refers to as a "charset".

1.1. Overview of URIs

URIs are characterized as follows: Uniform Uniformity provides several benefits. It allows different types of resource identifiers to be used in the same context, even when the mechanisms used to access those resources may differ. It allows uniform semantic interpretation of common syntactic conventions across different types of resource identifiers. It allows introduction of new types of resource identifiers without interfering with the way that existing identifiers are used. It allows the identifiers to be reused in many different contexts, thus permitting new applications or protocols to leverage a pre- existing, large, and widely used set of resource identifiers.
Top   ToC   RFC3986 - Page 5
   Resource

      This specification does not limit the scope of what might be a
      resource; rather, the term "resource" is used in a general sense
      for whatever might be identified by a URI.  Familiar examples
      include an electronic document, an image, a source of information
      with a consistent purpose (e.g., "today's weather report for Los
      Angeles"), a service (e.g., an HTTP-to-SMS gateway), and a
      collection of other resources.  A resource is not necessarily
      accessible via the Internet; e.g., human beings, corporations, and
      bound books in a library can also be resources.  Likewise,
      abstract concepts can be resources, such as the operators and
      operands of a mathematical equation, the types of a relationship
      (e.g., "parent" or "employee"), or numeric values (e.g., zero,
      one, and infinity).

   Identifier

      An identifier embodies the information required to distinguish
      what is being identified from all other things within its scope of
      identification.  Our use of the terms "identify" and "identifying"
      refer to this purpose of distinguishing one resource from all
      other resources, regardless of how that purpose is accomplished
      (e.g., by name, address, or context).  These terms should not be
      mistaken as an assumption that an identifier defines or embodies
      the identity of what is referenced, though that may be the case
      for some identifiers.  Nor should it be assumed that a system
      using URIs will access the resource identified: in many cases,
      URIs are used to denote resources without any intention that they
      be accessed.  Likewise, the "one" resource identified might not be
      singular in nature (e.g., a resource might be a named set or a
      mapping that varies over time).

   A URI is an identifier consisting of a sequence of characters
   matching the syntax rule named <URI> in Section 3.  It enables
   uniform identification of resources via a separately defined
   extensible set of naming schemes (Section 3.1).  How that
   identification is accomplished, assigned, or enabled is delegated to
   each scheme specification.

   This specification does not place any limits on the nature of a
   resource, the reasons why an application might seek to refer to a
   resource, or the kinds of systems that might use URIs for the sake of
   identifying resources.  This specification does not require that a
   URI persists in identifying the same resource over time, though that
   is a common goal of all URI schemes.  Nevertheless, nothing in this
Top   ToC   RFC3986 - Page 6
   specification prevents an application from limiting itself to
   particular types of resources, or to a subset of URIs that maintains
   characteristics desired by that application.

   URIs have a global scope and are interpreted consistently regardless
   of context, though the result of that interpretation may be in
   relation to the end-user's context.  For example, "http://localhost/"
   has the same interpretation for every user of that reference, even
   though the network interface corresponding to "localhost" may be
   different for each end-user: interpretation is independent of access.
   However, an action made on the basis of that reference will take
   place in relation to the end-user's context, which implies that an
   action intended to refer to a globally unique thing must use a URI
   that distinguishes that resource from all other things.  URIs that
   identify in relation to the end-user's local context should only be
   used when the context itself is a defining aspect of the resource,
   such as when an on-line help manual refers to a file on the end-
   user's file system (e.g., "file:///etc/hosts").

1.1.1. Generic Syntax

Each URI begins with a scheme name, as defined in Section 3.1, that refers to a specification for assigning identifiers within that scheme. As such, the URI syntax is a federated and extensible naming system wherein each scheme's specification may further restrict the syntax and semantics of identifiers using that scheme. This specification defines those elements of the URI syntax that are required of all URI schemes or are common to many URI schemes. It thus defines the syntax and semantics needed to implement a scheme- independent parsing mechanism for URI references, by which the scheme-dependent handling of a URI can be postponed until the scheme-dependent semantics are needed. Likewise, protocols and data formats that make use of URI references can refer to this specification as a definition for the range of syntax allowed for all URIs, including those schemes that have yet to be defined. This decouples the evolution of identification schemes from the evolution of protocols, data formats, and implementations that make use of URIs. A parser of the generic URI syntax can parse any URI reference into its major components. Once the scheme is determined, further scheme-specific parsing can be performed on the components. In other words, the URI generic syntax is a superset of the syntax of all URI schemes.
Top   ToC   RFC3986 - Page 7

1.1.2. Examples

The following example URIs illustrate several URI schemes and variations in their common syntax components: ftp://ftp.is.co.za/rfc/rfc1808.txt http://www.ietf.org/rfc/rfc2396.txt ldap://[2001:db8::7]/c=GB?objectClass?one mailto:John.Doe@example.com news:comp.infosystems.www.servers.unix tel:+1-816-555-1212 telnet://192.0.2.16:80/ urn:oasis:names:specification:docbook:dtd:xml:4.1.2

1.1.3. URI, URL, and URN

A URI can be further classified as a locator, a name, or both. The term "Uniform Resource Locator" (URL) refers to the subset of URIs that, in addition to identifying a resource, provide a means of locating the resource by describing its primary access mechanism (e.g., its network "location"). The term "Uniform Resource Name" (URN) has been used historically to refer to both URIs under the "urn" scheme [RFC2141], which are required to remain globally unique and persistent even when the resource ceases to exist or becomes unavailable, and to any other URI with the properties of a name. An individual scheme does not have to be classified as being just one of "name" or "locator". Instances of URIs from any given scheme may have the characteristics of names or locators or both, often depending on the persistence and care in the assignment of identifiers by the naming authority, rather than on any quality of the scheme. Future specifications and related documentation should use the general term "URI" rather than the more restrictive terms "URL" and "URN" [RFC3305].
Top   ToC   RFC3986 - Page 8

1.2. Design Considerations

1.2.1. Transcription

The URI syntax has been designed with global transcription as one of its main considerations. A URI is a sequence of characters from a very limited set: the letters of the basic Latin alphabet, digits, and a few special characters. A URI may be represented in a variety of ways; e.g., ink on paper, pixels on a screen, or a sequence of character encoding octets. The interpretation of a URI depends only on the characters used and not on how those characters are represented in a network protocol. The goal of transcription can be described by a simple scenario. Imagine two colleagues, Sam and Kim, sitting in a pub at an international conference and exchanging research ideas. Sam asks Kim for a location to get more information, so Kim writes the URI for the research site on a napkin. Upon returning home, Sam takes out the napkin and types the URI into a computer, which then retrieves the information to which Kim referred. There are several design considerations revealed by the scenario: o A URI is a sequence of characters that is not always represented as a sequence of octets. o A URI might be transcribed from a non-network source and thus should consist of characters that are most likely able to be entered into a computer, within the constraints imposed by keyboards (and related input devices) across languages and locales. o A URI often has to be remembered by people, and it is easier for people to remember a URI when it consists of meaningful or familiar components. These design considerations are not always in alignment. For example, it is often the case that the most meaningful name for a URI component would require characters that cannot be typed into some systems. The ability to transcribe a resource identifier from one medium to another has been considered more important than having a URI consist of the most meaningful of components. In local or regional contexts and with improving technology, users might benefit from being able to use a wider range of characters; such use is not defined by this specification. Percent-encoded octets (Section 2.1) may be used within a URI to represent characters outside the range of the US-ASCII coded character set if this
Top   ToC   RFC3986 - Page 9
   representation is allowed by the scheme or by the protocol element in
   which the URI is referenced.  Such a definition should specify the
   character encoding used to map those characters to octets prior to
   being percent-encoded for the URI.

1.2.2. Separating Identification from Interaction

A common misunderstanding of URIs is that they are only used to refer to accessible resources. The URI itself only provides identification; access to the resource is neither guaranteed nor implied by the presence of a URI. Instead, any operation associated with a URI reference is defined by the protocol element, data format attribute, or natural language text in which it appears. Given a URI, a system may attempt to perform a variety of operations on the resource, as might be characterized by words such as "access", "update", "replace", or "find attributes". Such operations are defined by the protocols that make use of URIs, not by this specification. However, we do use a few general terms for describing common operations on URIs. URI "resolution" is the process of determining an access mechanism and the appropriate parameters necessary to dereference a URI; this resolution may require several iterations. To use that access mechanism to perform an action on the URI's resource is to "dereference" the URI. When URIs are used within information retrieval systems to identify sources of information, the most common form of URI dereference is "retrieval": making use of a URI in order to retrieve a representation of its associated resource. A "representation" is a sequence of octets, along with representation metadata describing those octets, that constitutes a record of the state of the resource at the time when the representation is generated. Retrieval is achieved by a process that might include using the URI as a cache key to check for a locally cached representation, resolution of the URI to determine an appropriate access mechanism (if any), and dereference of the URI for the sake of applying a retrieval operation. Depending on the protocols used to perform the retrieval, additional information might be supplied about the resource (resource metadata) and its relation to other resources. URI references in information retrieval systems are designed to be late-binding: the result of an access is generally determined when it is accessed and may vary over time or due to other aspects of the interaction. These references are created in order to be used in the future: what is being identified is not some specific result that was obtained in the past, but rather some characteristic that is expected to be true for future results. In such cases, the resource referred to by the URI is actually a sameness of characteristics as observed
Top   ToC   RFC3986 - Page 10
   over time, perhaps elucidated by additional comments or assertions
   made by the resource provider.

   Although many URI schemes are named after protocols, this does not
   imply that use of these URIs will result in access to the resource
   via the named protocol.  URIs are often used simply for the sake of
   identification.  Even when a URI is used to retrieve a representation
   of a resource, that access might be through gateways, proxies,
   caches, and name resolution services that are independent of the
   protocol associated with the scheme name.  The resolution of some
   URIs may require the use of more than one protocol (e.g., both DNS
   and HTTP are typically used to access an "http" URI's origin server
   when a representation isn't found in a local cache).

1.2.3. Hierarchical Identifiers

The URI syntax is organized hierarchically, with components listed in order of decreasing significance from left to right. For some URI schemes, the visible hierarchy is limited to the scheme itself: everything after the scheme component delimiter (":") is considered opaque to URI processing. Other URI schemes make the hierarchy explicit and visible to generic parsing algorithms. The generic syntax uses the slash ("/"), question mark ("?"), and number sign ("#") characters to delimit components that are significant to the generic parser's hierarchical interpretation of an identifier. In addition to aiding the readability of such identifiers through the consistent use of familiar syntax, this uniform representation of hierarchy across naming schemes allows scheme-independent references to be made relative to that hierarchy. It is often the case that a group or "tree" of documents has been constructed to serve a common purpose, wherein the vast majority of URI references in these documents point to resources within the tree rather than outside it. Similarly, documents located at a particular site are much more likely to refer to other resources at that site than to resources at remote sites. Relative referencing of URIs allows document trees to be partially independent of their location and access scheme. For instance, it is possible for a single set of hypertext documents to be simultaneously accessible and traversable via each of the "file", "http", and "ftp" schemes if the documents refer to each other with relative references. Furthermore, such document trees can be moved, as a whole, without changing any of the relative references. A relative reference (Section 4.2) refers to a resource by describing the difference within a hierarchical name space between the reference context and the target URI. The reference resolution algorithm,
Top   ToC   RFC3986 - Page 11
   presented in Section 5, defines how such a reference is transformed
   to the target URI.  As relative references can only be used within
   the context of a hierarchical URI, designers of new URI schemes
   should use a syntax consistent with the generic syntax's hierarchical
   components unless there are compelling reasons to forbid relative
   referencing within that scheme.

      NOTE: Previous specifications used the terms "partial URI" and
      "relative URI" to denote a relative reference to a URI.  As some
      readers misunderstood those terms to mean that relative URIs are a
      subset of URIs rather than a method of referencing URIs, this
      specification simply refers to them as relative references.

   All URI references are parsed by generic syntax parsers when used.
   However, because hierarchical processing has no effect on an absolute
   URI used in a reference unless it contains one or more dot-segments
   (complete path segments of "." or "..", as described in Section 3.3),
   URI scheme specifications can define opaque identifiers by
   disallowing use of slash characters, question mark characters, and
   the URIs "scheme:." and "scheme:..".

1.3. Syntax Notation

This specification uses the Augmented Backus-Naur Form (ABNF) notation of [RFC2234], including the following core ABNF syntax rules defined by that specification: ALPHA (letters), CR (carriage return), DIGIT (decimal digits), DQUOTE (double quote), HEXDIG (hexadecimal digits), LF (line feed), and SP (space). The complete URI syntax is collected in Appendix A.

2. Characters

The URI syntax provides a method of encoding data, presumably for the sake of identifying a resource, as a sequence of characters. The URI characters are, in turn, frequently encoded as octets for transport or presentation. This specification does not mandate any particular character encoding for mapping between URI characters and the octets used to store or transmit those characters. When a URI appears in a protocol element, the character encoding is defined by that protocol; without such a definition, a URI is assumed to be in the same character encoding as the surrounding text. The ABNF notation defines its terminal values to be non-negative integers (codepoints) based on the US-ASCII coded character set [ASCII]. Because a URI is a sequence of characters, we must invert that relation in order to understand the URI syntax. Therefore, the
Top   ToC   RFC3986 - Page 12
   integer values used by the ABNF must be mapped back to their
   corresponding characters via US-ASCII in order to complete the syntax
   rules.

   A URI is composed from a limited set of characters consisting of
   digits, letters, and a few graphic symbols.  A reserved subset of
   those characters may be used to delimit syntax components within a
   URI while the remaining characters, including both the unreserved set
   and those reserved characters not acting as delimiters, define each
   component's identifying data.

2.1. Percent-Encoding

A percent-encoding mechanism is used to represent a data octet in a component when that octet's corresponding character is outside the allowed set or is being used as a delimiter of, or within, the component. A percent-encoded octet is encoded as a character triplet, consisting of the percent character "%" followed by the two hexadecimal digits representing that octet's numeric value. For example, "%20" is the percent-encoding for the binary octet "00100000" (ABNF: %x20), which in US-ASCII corresponds to the space character (SP). Section 2.4 describes when percent-encoding and decoding is applied. pct-encoded = "%" HEXDIG HEXDIG The uppercase hexadecimal digits 'A' through 'F' are equivalent to the lowercase digits 'a' through 'f', respectively. If two URIs differ only in the case of hexadecimal digits used in percent-encoded octets, they are equivalent. For consistency, URI producers and normalizers should use uppercase hexadecimal digits for all percent- encodings.

2.2. Reserved Characters

URIs include components and subcomponents that are delimited by characters in the "reserved" set. These characters are called "reserved" because they may (or may not) be defined as delimiters by the generic syntax, by each scheme-specific syntax, or by the implementation-specific syntax of a URI's dereferencing algorithm. If data for a URI component would conflict with a reserved character's purpose as a delimiter, then the conflicting data must be percent-encoded before the URI is formed.
Top   ToC   RFC3986 - Page 13
      reserved    = gen-delims / sub-delims

      gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"

      sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
                  / "*" / "+" / "," / ";" / "="

   The purpose of reserved characters is to provide a set of delimiting
   characters that are distinguishable from other data within a URI.
   URIs that differ in the replacement of a reserved character with its
   corresponding percent-encoded octet are not equivalent.  Percent-
   encoding a reserved character, or decoding a percent-encoded octet
   that corresponds to a reserved character, will change how the URI is
   interpreted by most applications.  Thus, characters in the reserved
   set are protected from normalization and are therefore safe to be
   used by scheme-specific and producer-specific algorithms for
   delimiting data subcomponents within a URI.

   A subset of the reserved characters (gen-delims) is used as
   delimiters of the generic URI components described in Section 3.  A
   component's ABNF syntax rule will not use the reserved or gen-delims
   rule names directly; instead, each syntax rule lists the characters
   allowed within that component (i.e., not delimiting it), and any of
   those characters that are also in the reserved set are "reserved" for
   use as subcomponent delimiters within the component.  Only the most
   common subcomponents are defined by this specification; other
   subcomponents may be defined by a URI scheme's specification, or by
   the implementation-specific syntax of a URI's dereferencing
   algorithm, provided that such subcomponents are delimited by
   characters in the reserved set allowed within that component.

   URI producing applications should percent-encode data octets that
   correspond to characters in the reserved set unless these characters
   are specifically allowed by the URI scheme to represent data in that
   component.  If a reserved character is found in a URI component and
   no delimiting role is known for that character, then it must be
   interpreted as representing the data octet corresponding to that
   character's encoding in US-ASCII.

2.3. Unreserved Characters

Characters that are allowed in a URI but do not have a reserved purpose are called unreserved. These include uppercase and lowercase letters, decimal digits, hyphen, period, underscore, and tilde. unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
Top   ToC   RFC3986 - Page 14
   URIs that differ in the replacement of an unreserved character with
   its corresponding percent-encoded US-ASCII octet are equivalent: they
   identify the same resource.  However, URI comparison implementations
   do not always perform normalization prior to comparison (see Section
   6).  For consistency, percent-encoded octets in the ranges of ALPHA
   (%41-%5A and %61-%7A), DIGIT (%30-%39), hyphen (%2D), period (%2E),
   underscore (%5F), or tilde (%7E) should not be created by URI
   producers and, when found in a URI, should be decoded to their
   corresponding unreserved characters by URI normalizers.

2.4. When to Encode or Decode

Under normal circumstances, the only time when octets within a URI are percent-encoded is during the process of producing the URI from its component parts. This is when an implementation determines which of the reserved characters are to be used as subcomponent delimiters and which can be safely used as data. Once produced, a URI is always in its percent-encoded form. When a URI is dereferenced, the components and subcomponents significant to the scheme-specific dereferencing process (if any) must be parsed and separated before the percent-encoded octets within those components can be safely decoded, as otherwise the data may be mistaken for component delimiters. The only exception is for percent-encoded octets corresponding to characters in the unreserved set, which can be decoded at any time. For example, the octet corresponding to the tilde ("~") character is often encoded as "%7E" by older URI processing implementations; the "%7E" can be replaced by "~" without changing its interpretation. Because the percent ("%") character serves as the indicator for percent-encoded octets, it must be percent-encoded as "%25" for that octet to be used as data within a URI. Implementations must not percent-encode or decode the same string more than once, as decoding an already decoded string might lead to misinterpreting a percent data octet as the beginning of a percent-encoding, or vice versa in the case of percent-encoding an already percent-encoded string.

2.5. Identifying Data

URI characters provide identifying data for each of the URI components, serving as an external interface for identification between systems. Although the presence and nature of the URI production interface is hidden from clients that use its URIs (and is thus beyond the scope of the interoperability requirements defined by this specification), it is a frequent source of confusion and errors in the interpretation of URI character issues. Implementers have to be aware that there are multiple character encodings involved in the
Top   ToC   RFC3986 - Page 15
   production and transmission of URIs: local name and data encoding,
   public interface encoding, URI character encoding, data format
   encoding, and protocol encoding.

   Local names, such as file system names, are stored with a local
   character encoding.  URI producing applications (e.g., origin
   servers) will typically use the local encoding as the basis for
   producing meaningful names.  The URI producer will transform the
   local encoding to one that is suitable for a public interface and
   then transform the public interface encoding into the restricted set
   of URI characters (reserved, unreserved, and percent-encodings).
   Those characters are, in turn, encoded as octets to be used as a
   reference within a data format (e.g., a document charset), and such
   data formats are often subsequently encoded for transmission over
   Internet protocols.

   For most systems, an unreserved character appearing within a URI
   component is interpreted as representing the data octet corresponding
   to that character's encoding in US-ASCII.  Consumers of URIs assume
   that the letter "X" corresponds to the octet "01011000", and even
   when that assumption is incorrect, there is no harm in making it.  A
   system that internally provides identifiers in the form of a
   different character encoding, such as EBCDIC, will generally perform
   character translation of textual identifiers to UTF-8 [STD63] (or
   some other superset of the US-ASCII character encoding) at an
   internal interface, thereby providing more meaningful identifiers
   than those resulting from simply percent-encoding the original
   octets.

   For example, consider an information service that provides data,
   stored locally using an EBCDIC-based file system, to clients on the
   Internet through an HTTP server.  When an author creates a file with
   the name "Laguna Beach" on that file system, the "http" URI
   corresponding to that resource is expected to contain the meaningful
   string "Laguna%20Beach".  If, however, that server produces URIs by
   using an overly simplistic raw octet mapping, then the result would
   be a URI containing "%D3%81%87%A4%95%81@%C2%85%81%83%88".  An
   internal transcoding interface fixes this problem by transcoding the
   local name to a superset of US-ASCII prior to producing the URI.
   Naturally, proper interpretation of an incoming URI on such an
   interface requires that percent-encoded octets be decoded (e.g.,
   "%20" to SP) before the reverse transcoding is applied to obtain the
   local name.

   In some cases, the internal interface between a URI component and the
   identifying data that it has been crafted to represent is much less
   direct than a character encoding translation.  For example, portions
   of a URI might reflect a query on non-ASCII data, or numeric
Top   ToC   RFC3986 - Page 16
   coordinates on a map.  Likewise, a URI scheme may define components
   with additional encoding requirements that are applied prior to
   forming the component and producing the URI.

   When a new URI scheme defines a component that represents textual
   data consisting of characters from the Universal Character Set [UCS],
   the data should first be encoded as octets according to the UTF-8
   character encoding [STD63]; then only those octets that do not
   correspond to characters in the unreserved set should be percent-
   encoded.  For example, the character A would be represented as "A",
   the character LATIN CAPITAL LETTER A WITH GRAVE would be represented
   as "%C3%80", and the character KATAKANA LETTER A would be represented
   as "%E3%82%A2".



(page 16 continued on part 2)

Next Section