tech-invite   World Map     

IETF     RFCs     Groups     SIP     ABNFs    |    3GPP     Specs     Glossaries     Architecture     IMS     UICC    |    search

RFC 2967

 
 
 

TISDAG - Technical Infrastructure for Swedish Directory Access Gateways

Part 2 of 5, p. 15 to 42
Prev RFC Part       Next RFC Part

 


prevText      Top      Up      ToC       Page 15 
4.0 Architecture

4.1 Software Components

   The conceptual architecture of the DAG is represented in Figure 4.1.
   General architectural specifications are described below, followed by
   individual component specifications Sections 5.5 through 5.12.

4.1.1 Internal Communications

   Communications between components of the DAG  will be by TCP/IP
   connections, using the DAG-Internal Protocol (DAG/IP).  DAG/IP is
   used by DAG-CAPs to communicate with the Referral Index and DAG-SAPs.
   Thus, the DAG/IP defines

   - the DAG-CAPs' range of query ability in the Referral Index (to
     gather referrals in response to the end-user's requests)
   - the responses (and their formats) of the Referral Index to the
     DAG-CAP requests
   - the DAG-CAPs' range of query ability to the DAG-SAPs for pursuing
     referrals when the DAG-CAP needs to do chaining for the client
     access software
   - the responses (and their formats) of the DAG-SAPs to the DAG-CAPs.

   The detail of the planned DAG/IP is given in Appendix C.  The detail
   of the DAG-CAP--Referral Index and DAG-CAP--DAG-SAP interactions  is
   given in the definitions of individual DAG-CAPs and DAG-SAPs, below
   (Sections 5.5 through 5.12).

4.1.2 Referral Index

   The Referral Index is responsible for maintaining the index of WDSP
   information, and providing a list of reasonable referrals in response
   to DAG-CAP search requests.  These "referrals" provide pointers to
   identify WDSPs that may have information that matches the end-user's
   query.

4.1.3 DAG-CAPs

   Individual DAG-CAPs are responsible for providing a particular client
   access protocol interface to the DAG service.  DAG-CAPs receive end-
   user queries in a particular query access protocol, convert the
   request into a query for the Referral Index ( i.e., expressed in
   DAG/IP), and then convert the Referral Index's response into a form
   that is appropriate for the client access protocol.  This may mean
   passing back the referrals directly, calling on DAG-SAPs to do the
   work of translating the referral into results ("chaining"), or a
   combination of both.

Top      Up      ToC       Page 16 
              +-------------------------------------+
              |+====+                               |
   HTTP   <-->+|    |<------+  (Full chaining)      |
              ||    |       |                       |
              |+====+       |                       |
              |             |                 +----+|
              |             |      Referral-->|    ||
              |             |      Result  <--|    |+<--> Whois++
              |             |                 +----+|
              |+====+       |                       |
   SMTP   <-->+|    |<------+  (Full chaining)      |
              ||    |       |                       |
              |+====+       |                       |
              |             |                 +----+|
              |             |      Referral-->|    ||
              |             |      Result  <--|    |+<--> LDAPv2
              |             |                 +----+|
              |+====+       |                       |
   Whois++<-->+|    |<------+  (Chain LDAPv2/3)     |
              ||    |       |                       |
              |+====+       |                       |
              |             |                 +----+|
              |             |      Referral-->|    ||
              |             |      Result  <--|    |+<--> LDAPv3
              |             |                 +----+|
              |+====+       |                       |
   LDAPv2 <-->+|    |<------+  (Full chaining)      |
              ||    |       |                       |
              |+====+       |                       |
              |             |                       |
              |+====+       |                       |
   LDAPv3 <-->+|    |<------+  (Chain Whois++)      |
              ||    |       |                       |
              |+====+       |                       |
              |             |                       |
              |             v                       |
              |   +-----------------------+         |
              |   |  Referral Index       |<---------------> Common
              |   |                       |         | Indexing Protocol
              |   +-----------------------+         | (CIP)
              +-------------------------------------+

            All internal communications are in DAG/IP.

            Figure 4.1 Conceptual Architecture of the DAG

Top      Up      ToC       Page 17 
4.1.4 DAG-SAPs

   Individual DAG-SAPs are called upon (by DAG-CAPs) to take DAG-
   generated referrals and pursue them -- issuing the indicated query at
   the specified WDSP service.  Results from individual WDSPs are
   converted back into DAG/IP-specific format for the DAG-CAP that made
   the request.  Each DAG-SAP is responsible for handling referrals to
   WDSPs of a particular protocol (e.g., LDAPv2, Whois++, etc).

4.2 Important Architectural Notes

   This section notes some of the thinking that has driven the
   architectural and software design specification for the DAG system.
   This helps to provide the context in which to understand the software
   specifications that follow, and should give clues for the eventual
   extension of the DAG system.  This section also acts, in some ways,
   as an FAQ (Frequently Asked Questions) section, as the content is
   shaped by questions received during the tech spec development phase.
   It attempts to illuminate context that may not otherwise be apparent
   on a first reading of the software specifications.

4.2.1 2 Distinct Functions:  Referrals and Chaining

   At all times, it must be kept in mind that the primary function of
   the DAG system is to provide users with referrals to WDSP services
   that may have the information they seek.  Since it is the case that
   not all supported client protocols can handle referrals, the DAG
   system also provides a chaining service to pursue referrals that the
   user's client software cannot handle itself.  This chaining service
   does attempt to match the user's query against data from WDSPs, but
   this is to be seen as a secondary, or support function of the DAG
   system.  In the perfect future, all access protocols will be able to
   handle all referrals!

4.2.2 Limited Query and Response Semantics

   The DAG system does not attempt to be a chameleon, or the ultimate
   whitepages query service.  It focuses on providing referrals for
   information on the limited number of query types outlined in the
   functional specifications of the DAG service.  This makes the DAG
   system a good place to start a search, but refinements and detailed
   inquiries are beyond its scope.

4.2.3 Visibility

   Given the limited query syntax of the DAG system it will not always
   be possible to exactly match a query posed to a CAP into a query
   posed to a SAP.  This will have the effect that for instance a LDAPv2

Top      Up      ToC       Page 18 
   client that issues a query to the DAG system which by the DAG system
   is chained to a LDAP server might not get the same results as if the
   client where directly connected to the server in question.

4.2.4 Richness of Query semantics

   Even the limited query syntax of the DAG system is capable of
   expressing queries that might NOT be possible to represent in the
   access protocols to the WDSPs.  In these cases the DAG-SAP either can
   refuse the query or try to emulate it.

4.2.5 N+M Protocol Mappings

   As part of the chaining service offered by the DAG system, a certain
   amount of mapping between protocols is required -- in theoretical
   terms, there  are "N" allowable end-user query access protocols, and
   "M" supported WDSP server protocols.  The architecture of the
   software is constructed to use a single internal protocol (the
   DAG/IP) and data schema, providing a common language between all
   components.  Without this, each input protocol module (DAG-CAP) would
   have to be constructed to be able to handle every WDSP protocol --
   NxM protocol mappings.  This would make the system complex, and
   difficult to expand to include new protocols in future.

4.2.6 DAG-CAPs and DAG-SAPs are completely independent of each other

   For the above reasons, the DAG-CAP and DAG-SAP modules are intended
   to be completely independent of each other.  A DAG-SAP responds to a
   query that is posed to it in the DAG/IP, without regard to the
   protocol of the DAG-CAP that passed the query.

4.2.7 The Role of the DAG-CAP

   Thus, the DAG-CAP is responsible for using the DAG/IP to obtain
   referral information and, where necessary, chained responses.  Where
   necessary, it performs adjustments to accommodate the differences in
   semantics between the DAG/IP and its native protocol.  This might
   involved doing post-filtering of the results returned by the DAG-SAPs
   since the query issued in DAG/IP to the DAG-SAP might be "broader"
   then the original query.

   Thus, the DAG-CAP "knows" only 2 protocols:  its native protocol, and
   the DAG/IP.

Top      Up      ToC       Page 19 
4.2.8 The Role of the DAG-SAP

   Similarly, the DAG-SAP is responsible for responding to DAG/IP
   queries by contacting the designated WDSP server.  Where necessary,
   it performs adjustments to accommodate the differences in semantics
   between the DAG/IP and its native protocol.  These adjustments might
   mean that, as a consequence, the DAG-SAP will receive results that do
   not match the original query.  In such cases the DAG-SAP should
   attempt to do post-pruning in order to reduce the mismatch between
   the original query and the results returned.

   Thus, the DAG-SAP "knows" only 2 protocols:  its native protocol, and
   the DAG/IP.

4.2.9 DAG/IP is internal

   No module outside of the DAG system should be aware of the DAG/IP's
   construction.  End-users use the query protocols supported by DAG-
   CAPs; WDSPs are contacted using the query protocols supported in the
   DAG-SAPs.

4.2.10 Expectations

   The expectation is that the DAG system, although defined as a single
   construct, will operate by running modules on several different,
   perhaps widely distributed (in terms of geography and ownership),
   computers.  For this reason, the DAG/IP specified in such a way that
   it will operate on inter-machine communications.

4.2.11 Future Extensions

   The DAG system architecture was constructed with a specific view to
   extensibility.  At any time, an individual component may be improved
   (e.g., the Mail DAG-CAP may be given a different query interface)
   without disrupting the system.

   Additionally, future versions of the DAG system may support other
   access protocols -- for end-users, and for WDSPs.

5.0 Software Specifications

5.1 Notational Convention

   It is always a challenge to accurately represent text protocol in a
   printed document; when is a new line a "newline", and when is it an
   effect of the text formatter?

Top      Up      ToC       Page 20 
   In order to be adequately illustrated, this document includes many
   segments of protocol grammars, sample data, and sample input/output
   in a text protocol.  In order to distinguish newlines that are
   significant in a protocol, the symbol

   <NL>

   is used.  For example,

   This is an example of a very long line of input.  There is only one
   newline in it (at the end), in spite of the fact that this document
   shows it spanning several lines of text.<NL>

5.2 DAG-CAP Basics

5.2.1 Functionality

   Every DAG-CAP must support the full range of DAG queries, as defined
   in 3.3.1.

   Each DAG-CAP accepts queries in its native protocol.  Individual
   DAG-CAP definitions define the expected expression of the DAG queries
   in the native protocol.

   The DAG-CAP is then responsible for:

   - converting that expression into a query in the DAG/IP to obtain
     relevant referrals from the Referral Index.  This might mean that
     parts of the original query are disregarded (e.g., if the query
     included attributes not supported by the DAG application, or if the
     query algebra was not supported by the DAG application);
   - returning referrals in the client's native protocol, where
     possible;
   - expressing the client query to the necessary DAG-SAPs, given the
     limitations mentioned above, to chain those referrals not usefully
     expressible in the client's native protocol;
   - possibly doing post-filtering on the DAG-SAP results; and
   - converting the collected DAG-SAP results for expression in the
     client's native protocol (and schema, where applicable).

   Each DAG-CAP defines the nature of the interaction with the end-user
   (e.g., synchronous or asynchronous, etc).  Additionally, each DAG-CAP
   must be able to carry out the following, in order to permit load-
   limiting and load-balancing in the DAG system:

   - direct the client to a different DAG-CAP of the same type (for
     load-balancing)

Top      Up      ToC       Page 21 
   - decline to return results because too many referrals were generated
     (to discourage data-mining).  Ideally, this should include the
     generation of a message to refine the query in order to produce a
     more manageable number of referrals/replies.

   DAG-CAPs must be capable of accepting and respecting DAG-SAP service
   referrals (for DAG-SAP load-sharing).

   In protocols that permit it, the DAG-CAP should indicate to the end-
   user which services were unavailable for chaining referrals (i.e., to
   indicate there were parts of the search that could not be completed,
   and information might be missing).

      TISDAG: Any CAP that receives commands other than queries, like
      help, answers those on its own.  A CAP should not pass any system
      command on to the RI.

5.2.2 Configuration

   It must be possible to change the expected address of the DAG-CAP by
   configuration of the software (i.e., host and port, e-mail address,
   etc).

   For DAG-CAPs that need to access DAG-SAPs for query chaining, for
   each type (protocol) of DAG-SAP that is needed, the DAG-CAP must be
   configurable in terms of:

   - at least one known DAG-SAP of every necessary protocol to contact
   - for each DAG-SAP, the host and port of the DAG-SAP software

   The DAG-CAPs must also be configurable in terms of a maximum number
   of referrals to handle for a user transaction (i.e., to prevent data
   mining, the DAG-CAP will refuse to reply if the query is too general
   and too many hits are generated at the Referral Index).

   The DAG-CAP must be configurable in terms of alternate DAG-CAPs of
   the same type to which the end-user software may be directed if this
   one is too busy.

5.2.3 Error handling

   Apart from error conditions arising from the operation of the DAG-CAP
   itself, DAG-CAPs are responsible for communicating error conditions
   occurring elsewhere in the system that affect the outcome of the
   user's query (e.g., in the DAG-RI, or in one or more DAG-SAPs).

Top      Up      ToC       Page 22 
   If the DAG-CAP sends a query to the DAG-RI and receives an error
   message, it should attempt to match the the received DAG errorcode
   into its native access protocol's error codes.  The same action is
   appropriate when the DAG-CAP is "chaining" the query to one DAG-SAP.

   There are also occasions when the DAG-CAP may have to combine
   multiple errorcodes into a single expression to the user.  When the
   DAG-CAP is "chaining" the query through DAG-SAPs to one or more
   WDSPs, situations can arise when there is a mix of responsecodes from
   the DAG-SAPs.  If this happens, the DAG-CAP should try to forward
   information to the end-user software that is as specific as possible,
   for instance which of the WDSPs has not been able to fulfill the
   query and why.

   See Appendix D for more information concerning error condition
   message mappings.

5.2.4 Pruning of results

   Since there is no perfect match between the query syntaxes of the DAG
   system on one hand and the different access protocols that the DAG-
   CAPs and DAG-SAPs supports on the other, there will be situations
   where the results a DAG-CAP has to collect is "broader" then what
   would have been the case if there had been a perfect match.  This
   might have adverse effects on the system to the extent that
   administrative limits will "unnecessary" be exceeded on WDSPs or that
   the collected results exceeds the sizelimit of the DAG-CAP.

   Since the DAG-CAP is the only part of the DAG system that actually
   knows what the original query was, the DAG-CAP can prune the results
   received from the DAG-SAPs in such a way that the results presented
   to the client better matches the original question.

5.3 DAG-SAP Basics

5.3.1 Functionality

   Every DAG-SAP must support the full range of DAG queries, as defined
   in 3.3.1.  Results must be complete DAG schemas expressed in well-
   formed DAG/IP result formats (see Appendix C).  Each DAG-SAP accepts
   queries in DAG/IP and converts them to the native schema and protocol
   for which it is designed to proxy.

   The DAG-SAP is then responsible for

   - converting the query into the native schema and protocol of the
     WDSP to which the referral points.  (If the query is not
     representable in the native protocol, it must return an error

Top      Up      ToC       Page 23 
     message.  If it is emulatable, the DAG-SAP can attempt emulate it
     by posing a related query to the WDSP and post-pruning the results
     received);
   - contacting that WDSP, using the host, port, and protocol
     information provided in the referral;
   - negotiating the query with the remote WDSP;
   - accepting results from the WDSP, possibly doing post-filtering on
     the result set; and
   - conveying the results back to the calling DAG-CAP using the DAG/IP
     and its schema.

   Note that this implicitly means that the DAG-SAP is responsible for
   chaining and pursuing any referrals it receives from WDSP services.
   The DAG-SAP returns only search results to the DAG-CAP that called
   it.

5.3.2 Configuration

   DAG-SAPs must be configurable to accept connections only from
   recognized DAG components.

   DAG-SAPs that have service limits must be configurable to redirect
   DAG-CAPs to alternate DAG-SAPs of the same type when necessary.

5.3.3 Error handling

   A DAG-SAP must translate error codes received from a WDSP server to
   DAG error codes according to Appendix D.

5.3.4 Pruning of results

   Since it might not be possible to exactly map a DAG query into a
   query in the access protocol supported by the a DAG-SAP, the DAG-SAP
   should try to translate it into a more general query (or if necessary
   into a set of queries).  If so, the DAG-SAP must then prune the
   result set received before furthering it to the DAG-CAP.

5.3.5 Constraint precedence

   Some constraints, search and case, can appear both as local and
   global constraints.  If this happens in a query then the local
   constraint specification overrides the global.  For a query like the
   following:

   fn=leslie;search=exact and org=think:search=substring

   the resulting search constraint for "fn=leslie" will be "exact" while
   it for "org=think" will be "substring".

Top      Up      ToC       Page 24 
5.4 The Referral Index

5.4.1 Architecture

   The Referral Index contains (only) information necessary to deliver
   referrals to DAG-CAPs based on the query types supported by the DAG
   itself.  The Referral Index creates an index over these objects so
   that it can respond to DAG-CAP queries using the DAG/IP.  The
   information is drawn directly from interactions with participating
   WDSPs' software, using the Common Indexing Protocol (CIP).

5.4.2 Interactions with WDSPs (CIP)

   WDSPs that wish to participate in the DAG system must register
   themselves (see Section 5.4.6).  Once registered, the Referral Index
   will interact with the WDSPs using the Common Indexing Protocol as
   defined in [1], using the Index Object defined in Section 5.4.3.

5.4.3 Index Object Format

   The CIP index object type is based on the Tagged Index Object as
   defined in [12].  Appendix E details the expected content of the
   index objects as they are to be provided by the WDSPs.

      TISDAG: The tokens in the Tagged Index Object should be UTF-8
      encoded composed UNICODE version 2 character encoding.

5.4.4 DAG-Internal I/O

   The Referral Index interacts with the rest of the DAG internal
   modules (DAG-CAPs) by listening for queries and responding in the
   DAG/IP (defined in Appendix C).

5.4.5 The Index Server

   The Referral Index must index the necessary attributes of the CIP
   index object in order to respond to queries of the form described in
   Table 3.1.

   The semantics of the chosen CIP object (defined in Appendix E) are
   such that a referral to a WDSP server is sent back if (and only if)

   - the index object of the WDSP contains all the tokens of the query,
     in the attributes specified, according to the logic of the DAG/IP
     query, and
   - all of those tokens are found with a common tag.

Top      Up      ToC       Page 25 
   This means that a query for the name "Fred Flintstone" (2 tokens)
   will yield a referral to a server that has a record for "Fred Amadeus
   Flintstone", but not to a WDSP with 2 differently tagged records, for
   "Fred Amadeus" and "Julie Flintstone".  Depending on the access
   protocol being used and the original end-user query, the referral to
   the WDSP with "Fred Amadeus Flintstone" may yield a successful
   result, or it may not.  But, it is known that the other WDSP would
   not have yielded successful searches.  That is, the referral approach
   may yield false-positive results, but will not miss appropriate
   WDSPs.

5.4.6 Configuration

   The Referral Index must provide the ability to register interested
   WDSPs, as outlined in Appendix E.

   The Referral Index must be able to configure the port for DAG/IP
   communications.  Also, it must be configurable to recognize only
   registered DAG-CAPs.

5.4.7 Security

   The Referral Index will accept queries only from recognized
   (registered) DAG-CAPs.  This will reduce "denial of service" attack
   types, but is also a reflection on the fact that the Referral Index
   uses the DAG/IP, (i.e., internal) protocol, which should not be
   exposed to non-DAG software.

   The Referral Index must be able to use authenticated communication to
   receive data from WDSPs (see Appendix E).

5.5 Mail (SMTP) DAG-CAP

   This is the default Mail DAG-CAP.  More sophisticated ones could
   certainly be written -- e.g., for pretty-printed output, or for
   handling different philosophies of case-matching.

   This DAG-CAP has been designed on the assumption that mail queries
   will be human-generated (i.e., using a mail program/text editor), as
   opposed to being queries formulated by software agents.  The input
   grammar should therefore be simple and liberal in acceptance of
   variations of whitespace formatting.

Top      Up      ToC       Page 26 
5.5.1 Mail DAG-CAP Input

   Mail DAG-CAP input is expected to be a regular or MIME-encoded (see
   [9] and [10]) SMTP mail message, sent to an advertised mail address.
   The mail DAG-CAP parses the message and replies to it with a MIME-
   encoded message containing the results of the DAG search.

   One query is accepted per e-mail message -- text after a single valid
   query has been read is simply ignored.

   The body of the query message must follow the syntax defined below.
   Note that all input control terms ("type=", "name=" etc) are shown in
   lower case for convenience, but could be upper case or mixed case on
   input.

   mailquery       = [mnl] [controls] mnl terms mnl
   controls        = [msp] "searchtype" [msp] "=" [msp]
                        ( matchtype /
                          casetype /
                          matchtype msp casetype /
                          casetype msp matchtype /
                          <nothing> )
   matchtype       = "substring" / "exact"
                  ; default:  substring
   casetype        = "ignore" / "sensitive"
                  ; default:  ignore

   terms           = n / n-l / n-o / n-o-l / r-o / r-o-l

   n               = n-term
   n-l             = ( n-term l-term  / l-term n-term)
   n-o             = ( n-term o-term  / o-term n-term )
   n-o-l           = ( n-term o-term l-term /
                    n-term l-term o-term /
                    l-term n-term o-term /
                    l-term o-term n-term /
                    o-term l-term n-term /
                    o-term n-term l-term )
   r-o             = ( r-term o-term / o-term r-term )
   r-o-l           = ( r-term o-term l-term /
                    r-term l-term o-term /

                    l-term o-term r-term /
                    l-term r-term o-term /
                    o-term l-term r-term /
                    o-term r-term l-term )
   n-term          = [msp] "name" [msp] "=" [msp] string mnl
   o-term          = [msp] "org" [msp] "=" [msp] string mnl

Top      Up      ToC       Page 27 
   l-term          = [msp] "loc" [msp] "=" [msp] string mnl
   r-term          = [msp] "role" [msp] "=" [msp] string mnl

   string          = <US-ASCII or quoted-printable encoded
                   ISO-8859-1 or UTF-8 except nl and sp>
   msp             = 1*(sp)
    sp              = " "
   mnl             = 1*(nl)

   nl              = <linebreak>

   The following are valid mail queries:

   Example 1:

   searchtype =   <NL>
   name = thinking cat<NL>

   Example 2:

   searchtype = exact ignore<NL>
   name=thinking cat<NL>

   Example 3:

   role=thinking cat<NL>
   org =space colonization<NL>

   Example 4:

   name=thinking cat <NL>
   <NL>
   <NL>
   My signature line follows here in the most annoying
   fashion <NL>

   Note that the following are not acceptable queries:

   Example 5:

   searchtype= exact substring <NL>
   name = thinking cat <NL>

   Example 6:

   name=thinking cat org= freedom fighters anonymous<NL>

Top      Up      ToC       Page 28 
   In Example 5, two conflicting searchtypes are given.  In Example 6,
   no linebreak follows the n-term.

5.5.2 Translation from Mail query to DAG/IP

   Querying the Referral Index

   A key element of translating from the Mail DAG-CAP input into the
   DAG/IP query format is to "tokenize" the input terms into single
   token elements for the DAG/IP query.  For example,  the n-term

   name= thinking cat<NL>

   is tokenized into 2 n-tokens:

   thinking
   cat

   which are then mapped into the following in the DAG/IP query (dag-n-
   terms):

   FN=thinking and FN=cat<NL>

   The same is true for all r-terms, l-terms and o-terms.  The primary
   steps in translating the mail input into a DAG/IP query are:

   translate quoted-printable encoding, if necessary
   translate base64 encoding, if necessary
   tokenize the strings for each term
   construct the DAG/IP query from the resulting components, as
   described in more detail below

   DAG/IP constraints are constructed from the searchtype information in
   the query.

   dag-matchtype = "search=" <matchtype> /
                "search=substring"  ; if matchtype not
                                    ; specified

   dag-casetype  = "case=ignore"  /    ; if casetype not
                                    ; specified or
                                    ; casetype=ignore
                "case=consider"     ; if casetype=sensitive

   constraints   = ":" dag-matchtype ";" dag-casetype

   The terms for the DAG/IP query are constructed from the tokenized
   strings from the mail input.

Top      Up      ToC       Page 29 
   dag-n-terms   = "FN=" n-token 0*( " and FN=" n-token)
   dag-o-terms   = "ORG=" o-token 0*( " and ORG=" o-token)
   dag-l-terms   = "LOC=" l-token 0*( " and LOC=" l-token)
   dag-r-terms   = "ROLE=" r-token 0*( " and ROLE=" r-token)

   This means that the relevant DAG/IP queries are formulated as one of
   two types:

   dagip-query   = ( ( ( n-query / nl-query / no-query /
                      nol-query ) [" and template=DAGPERSON"]":"
                   dag-matchtype ";" dag-casetype) /
                  ( ( ro-query / rol-query )
                    [" and template=DAGORGROLE"]":"
                    dag-matchtype ";" dag-casetype)  )

   n-query       = dag-n-terms
   nl-query      = dag-n-terms " and " dag-l-terms
   no-query      = dag-n-terms " and " dag-o-terms
   nol-query     = dag-n-terms " and " dag-o-terms " and "
                dag-l-terms
   ro-query      = dag-r-terms " and " dag-o-terms
   rol-query     = dag-r-terms " and " dag-o-terms " and "
                dag-l-terms

   The examples given earlier are then translated as follows.

   Example 1:

   FN=thinking and FN=cat:search=substring;case=ignore<NL>

   Example 2:

   FN=thinking and FN=cat:search=exact;case=ignore<NL>

   Example 3:

   ROLE=thinking and ROLE=cat and ORG=space and
   ORG=colonization:search=substring;case=ignore<NL>

   Querying a DAG-SAP

   In querying a DAG-SAP (irrespective of the protocol of that DAG-SAP),
   the DAG/IP query must include information about the target WDSP
   server.  This information is drawn from the Referral Index SERVER-
   TO-ASK referral information, and is appended to the query as
   specified in Appendix C):

Top      Up      ToC       Page 30 
   ":host=" quoted-hostname ";port=" number ";server-info="
   quoted-serverinfo ";charset=" charset

   where the response from the Referral Index included:

   "# SERVER-TO-ASK " serverhandle nl
   " Server-info: " serverinfo nl
   " Host-Name: " hostname nl
   " Host-Port: " number nl

   " Protocol: " prot nl
   " Source-URI: " source nl
   " Charset: " charset nl
   "# END" nl

   and the "quoted-hostname" and "quoted-serverinfo" are obtained from
   "hostname" and "serverinfo" respectively, by quoting the DAG/IP
   special characters.

   For example, the referral

   # SERVER-TO-ASK dagsystem01<NL>
    Server-info: o=thinkingcat, c=se<NL>
    Host-Name: thinkingcat.com<NL>
    Host-Port: 2839<NL>
    Protocol: ldapv2<NL>
    Source-URI: http://www.thinkcat.com
    Charset: T.61<NL>
    # END<NL>

   would yield the addition

   :host=thinkingcat\.com;port=2839;server-info=o\=thinkingcat\,\
   c\=se;charset=T\.61

   in its query to an LDAPv2 DAG-SAP.

   (N.B.: See Appendix C for further definitions of the terms used in
   the SERVER-TO-ASK response).

   Note that it is the DAG-SAP's responsibility to extract these terms
   from the query and use them to identify the WDSP server to be
   contacted.  See the individual DAG-SAP definitions, below.

Top      Up      ToC       Page 31 
5.5.3 Chaining queries in Mail DAG-CAP

   The Mail DAG-CAP has to chain all referrals -- to the Whois++ DAG-
   SAP, LDAPv2  DAG-SAP, or LDAPv3 DAG-SAP as appropriate for the
   referral.

5.5.4 Expression of results in Mail DAG-CAP

   The results message is sent to the "Reply-To:"  address of the
   originating mail, if available (see [4] for appropriate
   interpretation of mail originator headers).  The original query is
   repeated, along with the message-id.  The remainder of the body of
   the mail message is the concatenation of responses from the DAG-SAP
   calls, each result having the WDSP's SOURCE URI (from the referral)
   appended to it, and the system messages also having been removed.

   At the end of the message, the WDSP servers that failed to respond
   (i.e., the DAG-SAP handling the referral returned the "% 403
   Information Unavailable" message) are listed with their server-info.

5.5.5 Expression of Errors in Mail DAG-CAP

   If the mail DAG-CAP receives a message that is not parsable using the
   query grammar described above, it returns an explanatory message to
   the query mail's reply address saying that the query could not be
   interpreted, and giving a description of valid queries.

   If the number of referrals sent by the Referral Index is greater than
   the pre-determined maximum (for detecting data-mining efforts, or
   otherwise refusing over-general queries, such as "FN=svensson"), the
   mail DAG-CAP will send an explanatory message to the query mail's
   reply address describing the "over-generalized query" problem,
   suggesting the user resubmit a more precise query, and describing the
   list of valid query types.

   If the mail DAG-CAP receives several different result codes from the
   DAG-SAPs it should represent those in an appropriate manner in the
   response message.

   A mail DAG-CAP may redirect a connection to another mail DAG-CAP for
   reasons of load-balancing.  This is done simply by forwarding the
   mail query to the address of the alternate mail DAG-CAP.

Top      Up      ToC       Page 32 
5.6 Web (HTTP) DAG-CAP

5.6.1 Web DAG-CAP Input

   The web DAG-CAP provides its interface via standard HTTP protocol.
   The general expectation is that the web DAG-CAP will provide a form
   page with radio buttons to select "substring or exact match" and
   "consider case or ignore case".  Other information (about name, role,
   organization, locality) is solicited as free-form text.

   The DAG-CAP receives queries via an HTTP "post" method (the outcome
   of the form action for the page described above, or generated
   elsewhere).  The rest of this section describes the variables that
   are to be expressed in that post.  The actual layout of the page and
   most user interface issues are left to the discretion of the builder.
   Note that the Web DAG-CAP may be called upon to provide responses in
   different content encoding, and must therefore address the "Accept-
   Encoding:" request header in the HTTP connection.

   Although the Web protocol, HTTP, is not itself capable of handling
   referrals, through the use of two extra variables this client is
   given the option of requesting referral information and then pursuing
   individual referrals through the Web DAG-CAP itself, as a proxy for
   those referrals.  This is handled through the extra "control
   variables" to request referrals only, and to indicate when the
   transaction is a continuation of a previous query to pursue a
   referral.

   There has been call to have a "machine-readable" version of the
   search output.  As HTML is geared towards visual layout, user agents
   that intend to do something with the results other than present them
   in an HTML browser have few cues to use to extract the relevant
   information from the HTML page.  Also, "minor" visual changes,
   accomplished with extensive HTML updates, can disrupt user agents
   that were built to blindly parse the original HTML.  Therefore,
   provision has been made to return "raw" format results.  These are
   requested by specifying "Accept-Content: application/whoispp-
   response"  in the request header of the HTTP message to the HTTP
   DAG-CAP.

Top      Up      ToC       Page 33 
   The variables that are expected are:

   transaction     = "new" / "chain"  ; default is "new". This
                   ; should not be user-settable.  It is used
                   ; in constructed URLs
   resulttype      = "all" / "referrals" ; default is "all"
   matchtype       = "substring" / "exact"
   casetype        = "case ignore" / "case sensitive"
   n-term          = string
   o-term          = string
   l-term          = string
   r-term          = string
   host-term       = string
   port-term       = string
   servinfo-term   = string
   prot-term       = string ; the protocol of the referral
   string          = <UNICODE-2-0-UTF-8> / <UNICODE-1-1-UTF-8> /
                  <ISO-8859-1>

5.6.2 Translation from Web query to DAG/IP

   Querying a DAG-SAP Directly

   If the transaction variable is "chain", the information in the POST
   is used to pursue a particular referral, not do a search of the
   Referral Index.  The appropriate DAG-SAP (deduced from the prot-term)
   is contacted and issued the query directly.

   Results from this type of query are always full results (i.e., not
   referrals).

   Querying the Referral Index

   A key element of translating from the Web DAG-CAP input into the
   DAG/IP query format is to "tokenize" the input terms into single
   token elements for the DAG/IP query.  For example,  the n-term

   name= thinking cat

   is tokenized into 2 n-tokens:

   thinking
   cat

   which are then mapped into the following in the DAG/IP query (dag-n-
   terms):

   FN=thinking and FN=cat

Top      Up      ToC       Page 34 
   The same is true for the r-term, l-term and o-term.

   The primary steps in translating the HTTP input into a DAG/IP query
   are:

   translate encodings, if necessary
   tokenize the strings for each term
   construct the DAG/IP query from the resulting components, as
   described in more detail below

   DAG/IP constraints are constructed from the searchtype information in
   the query.

   dag-matchtype = "search=" <matchtype> /
                "search=substring"     ; if matchtype not
                                       ; specified

   dag-casetype  = "case=ignore"  /       ; if casetype not
                                       ; specified or
                                       ; casetype="case ignore"
                "case=consider"        ; if casetype=
                                       ; "case sensitive"

   constraints   = ":" dag-matchtype ";" dag-casetype

   The terms for the DAG/IP query are constructed from the tokenized
   strings from the HTTP post input.

   dag-n-terms   = "FN=" n-token 0*( " and FN=" n-token)
   dag-o-terms   = "ORG=" o-token 0*( " and ORG=" o-token)
   dag-l-terms   = "LOC=" l-token 0*( " and LOC=" l-token)
   dag-r-terms   = "ROLE=" r-token 0*( " and ROLE=" r-token)

   This means that the relevant DAG/IP queries are formulated as one of
   two types:

   dagip-query   = ( ( ( n-query / nl-query / no-query / nol-query )
                      [" and template=DAGPERSON"]":" dag-matchtype
                      ";" dag-casetype) /
                  ( ( ro-query / rol-query )
                      [" and template=DAGORGROLE"]":" dag-matchtype
                      ";" dag-casetype)  )

   n-query       = dag-n-terms

Top      Up      ToC       Page 35 
   nl-query      = dag-n-terms " and " dag-l-terms
   no-query      = dag-n-terms " and " dag-o-terms
   nol-query     = dag-n-terms " and " dag-o-terms " and "
                dag-l-terms
   ro-query      = dag-r-terms " and " dag-o-terms
   rol-query     = dag-r-terms " and " dag-o-terms " and "
                dag-l-terms

   Querying a DAG-SAP

   In querying a DAG-SAP (irrespective of the protocol of that DAG-SAP),
   the DAG/IP query must include information about the target WDSP
   server.  This information is drawn from the Referral Index SERVER-
   TO-ASK referral information, and is appended to the query as
   specified in Appendix C:

   ":host=" quoted-hostname ";port=" number ";server-info="
   quoted-serverinfo ";charset=" charset

   where the response from the Referral Index included:

   "# SERVER-TO-ASK " serverhandle <NL>
   " Server-info: " serverinfo <NL>
   " Host-Name: " hostname <NL>
   " Host-Port: " number <NL>
   " Protocol: " prot <NL>
   " Source-URI: " source <NL>
   " Charset: " charset <NL>
   "# END" <NL>

   and the "quoted-hostname" and "quoted-serverinfo" are obtained from
   "hostname" and "serverinfo" respectively, by quoting the DAG/IP
   special characters.

   For example, the referral

   # SERVER-TO-ASK dagsystem01<NL>
    Server-info: o=thinkingcat, c=se<NL>
    Host-Name: thinkingcat.com<NL>
    Host-Port: 2839<NL>
    Protocol: ldapv2<NL>
    Source-URI: http://www.thinkingcat.com
    Charset: T.61<NL>
   # END<NL>

   would yield the addition

Top      Up      ToC       Page 36 
   :host=thinkingcat\.com;port=2839;server-info=o\=thinkingcat\,\
   c\=se;charset=T\.61

   in its query to an LDAPv2 DAG-SAP

   (N.B.: See Appendix C for further definitions of the terms used in
   the SERVER-TO-ASK response).

   Note that it is the DAG-SAP's responsibility to extract these terms
   from the query and use them to identify the WDSP server to be
   contacted.  See the individual DAG-SAP definitions, below.

5.6.3 Chaining queries in Web DAG-CAP

   If the resulttype was "all", all of the referrals received from the
   Referral Index are chained using the appropriate DAG-SAPs.  If only
   referrals were requested, the Referral Index results are returned.

5.6.4 Expression of results in Web DAG-CAP

   text/html results

   The default response encoding is text/html.  If the resulttype was
   "all",  the content of the chaining responses from the DAG-SAPs,
   without the system messages, is collated into a single page response,
   one result entry per demarcated line ( e.g., bullet item).  The FN or
   ROLE value should be presented first and clearly.  The SOURCE URI for
   each WDSP referral should be presented as an HREF for each of the
   WDSPs results.

   At the end of the message, the WDSP servers that failed to respond
   (i.e., the DAG-SAP handling the referral returned the "% 403
   Information Unavailable" message) are listed with their server-info.

   If, however, the resulttype was "referrals", the results from the
   Referral Index are returned as HREF URLs to the Web DAG-CAP itself,
   with the necessary information to carry out the query (including the
   "HOST=", etc, for the referral).

   For example, if the original query:

   n-term="thinking cat"
   resulttype="referrals"

   drew the following referral from the Referral Index:

   # SERVER-TO-ASK DAG-Serverhandle<NL>
    Server-Info: c=se, o=tce<NL>

Top      Up      ToC       Page 37 
    Host-Name: answers.tce.com<NL>
    Host-Port: 1111<NL>

    Protocol: ldapv3<NL>
    Source-URI: http://some.service.se/
    Charset: UTF-8<NL>
   # END<NL>

   the response would be an HTML page with an HREF HTTP "POST" URL to
   the Web DAG-CAP with the following variables set:

   n-term="thinking cat"
   transaction="chain"
   servinfo-term="c=se, o=tce"
   host-term="answers.tce.com"
   port-term="1111"
   prot-term="ldapv3"

   The Source-URI should be established in the response as its own HREF
   URI.

   application/whoispp-response Results

   If Accept-Encoding: " HTTP request header had the value
   "application/whoispp-response", the content of the HTTP response will
   be constructed in the same syntax and attribute mapping as for the
   Whois++ DAG-CAP.

   If the resulttype was "all", all the referrals will have been chained
   by the Web DAG-CAP, and the response will include only full data
   records.

   If the resulttype was "referrals", then all referrals are passed
   directly back in a single response, in correct Whois++ referral
   format (conveniently, this is how they are formulated in the DAG/IP).
   Note that this will include referrals to LDAP-based services as well
   as Whois++ servers.

5.6.5 Expression of Errors in Web DAG-CAP

   A Web DAG-CAP may redirect a connection to another web DAG-CAP for
   reasons of load-balancing.  This is done simply by using an HTTP
   redirect.

Top      Up      ToC       Page 38 
   Standard Errors

   If the web DAG-CAP receives a message that is not parsable using the
   query grammar described above, it sends an explanatory HTML page
   saying that the query could not be interpreted, and giving a
   description of valid queries.

   If the number of referrals sent by the Referral Index is greater than
   the pre-determined maximum (for detecting data-mining efforts, or
   otherwise refusing over-general queries, such as "FN=svensson"), the
   web DAG-CAP will send a page with an explanatory message describing
   the "over-generalized query" problem, suggesting the user resubmit a
   more precise query, and describing the list of valid query types.

   If the web DAG-CAP receives more than one result code from the DAG-
   SAPs, it must represent them all in a appropriate manner in the
   response.

   application/whoispp-response Errors

   An invalid query is responded to with a simple text response with the
   error: "% 500 Syntax Error".

   If too many referrals are generated from the Referral Index, the
   simple text response will have the message "% 503 Query too general".

5.7 Whois++ DAG-CAP

      TISDAG: The system commands polled-for/-by should elicit the empty
      set as a return value until we better understand the implications
      of doing otherwise.

5.7.1 Whois++ DAG-CAP Input

   Input to the Whois++ DAG-CAP follows the Whois++ standard ([6]).
   Minimally, the Whois++ DAG-CAP must support the following queries:

   Query Type     Expression in Whois++
   -----------    ------------------------------------
   N              One or more "name=" and
                  template=USER

   NL             One or more "name=" and
                  One or more "address-locality=" and template=USER

   NO             One or more "name=" and
                  one or more "organization-name=" and template=USER

Top      Up      ToC       Page 39 
   NOL            One or more "name=" and
                  one or more  "organization-name=" and
                  one or more  "address-locality=" and template=USER

   RO             One or more "org-role=" and
                  one or more "organization-name=" and template=ORGROLE

   ROL            One or more "org-role=" and
                  one or more "organization-name=" and
                  one or more "address-locality=" and template=ORGROLE

      Table 5.1 Allowable Whois++ Queries

   The following constraints must be supported for queries:

   "search=" (substring / exact)
   "case=" (ignore / consider)

   If no constraints are defined in a query the default is exact and
   ignore.  For example,

   FN=foo and loc=kista and fn=bar<NL>

   is a perfectly valid Whois++ NL query for "Foo Bar" in "Kista".

5.7.2 Translation from Whois++ query to DAG/IP

   Querying the Referral Index

   The Whois++ DAG-CAP formulates a DAG/IP query by forwarding the
   search terms received (as defined in Table 5.1).

   For example, the above query would be expressed as:

   FN=foo and LOC=kista and FN=bar and template=DAGPERSON<NL>

   Querying a DAG-SAP

   In querying a DAG-SAP (irrespective of the protocol of that DAG-SAP),
   the DAG/IP query must include information about the target WDSP
   server.  This information is drawn from the Referral Index SERVER-
   TO-ASK referral information, and is appended to the query as
   specified in appendix C:

   ":host=" quoted-hostname ";port=" number ";server-info="
   quoted-serverinfo ";charset=" charset

Top      Up      ToC       Page 40 
   where the response from the Referral Index included:

   "# SERVER-TO-ASK " serverhandle<NL>
   " Server-info: " serverinfo<NL>
   " Host-Name: " hostname<NL>
   " Host-Port: " number<NL>
   " Protocol: " prot<NL>
   " Source-URI: " source<NL>
   " Charset: " charset<NL>
   "# END"<NL>

   and the "quoted-hostname" and "quoted-serverinfo" are obtained from
   "hostname" and "serverinfo" respectively, by quoting the DAG/IP
   special characters.

   For example, the referral

   # SERVER-TO-ASK dagsystem01<NL>
    Server-info: o=thinkingcat, c=se<NL>
    Host-Name: thinkingcat.com<NL>
    Host-Port: 2839<NL>
    Protocol: ldapv2<NL>
    Source-URI: http://www.thinkingcat.com/
    Charset: T.61<NL>
   # END<NL>

   would yield the addition

   :host=thinkingcat\.com;port=2839;server-info=o\=thinkingcat\,\
   c\=se;charset=T\.61

   in its query to an LDAPv2 DAG-SAP.

   (N.B.: See Appendix C for further definitions of the terms used in
   the SERVER-TO-ASK response).

   Note that it is the DAG-SAP's responsibility to extract these terms
   from the query and use them to identify the WDSP server to be
   contacted.  See the individual DAG-SAP definitions, below.

5.7.3 Chaining in Whois++ DAG-CAP

   The Whois++ DAG-CAP relies on DAG-SAPs to chain any non-Whois++
   referrals (currently, the LDAPv2 and LDAPv3 DAG-SAPs).

Top      Up      ToC       Page 41 
5.7.4 Expression of results in Whois++

   Results are expressed in Whois++ by collating the DAG/IP results
   received from DAG-SAPs (using the FULL response), and using the
   template and attribute mappings defined in Appendix B. For each
   result from a given referral, the SOURCE attribute is added, with the
   value of the SOURCE-URI from the referral.

   Any referrals to other Whois++ servers provided by the Referral Index
   are sent directly to the Whois++ client as follows:

   server-to-ask   =   "# SERVER-TO-ASK " DAG-Serverhandle<NL>
                    " Server-Handle: " SERVER-INFO<NL>
                    " Host-Name: " HOST<NL>
                    " Host-Port: " PORT<NL>
                    " Protocol: " PROTOCOL<NL>
                    "# END"<NL>

   where SERVER-INFO, HOST, PORT, PROTOCOL are drawn from the referral
   provided in the DAG/IP, and the SOURCE-URI information is lost.

5.7.5 Expression of Errors in Whois++ DAG-CAP

   As appropriate, the Whois++ DAG-CAP will express operational errors
   following the Whois++ standard.  There are 4 particular error
   conditions of the DAG system that the DAG-CAP will handle as
   described below.

   When the Whois++ DAG-CAP receives a query that it cannot reply to
   within the (data) constraints of the DAG, it sends an error message
   and closes the connection.  The error message includes

   % 502 Search expression too complicated<NL>

   If the number of referrals sent by the Referral Index is greater than
   the pre-determined maximum (for detecting data-mining efforts, or
   otherwise refusing over-general queries, such as "FN=svensson"), the
   Whois++ DAG-CAP will send an error message and close the connection.
   The error message includes

   % 503 Query too general<NL>

   (N.B.: this is different from the "Too many hits" reply, which does
   send partial results.)

Top      Up      ToC       Page 42 
   A Whois++ DAG-CAP may redirect a connection to another Whois++ DAG-
   CAP for reasons of load-balancing.  This is expressed to the end-user
   client software using the SERVER-TO-ASK response with appropriate
   information to reach the designated alternate DAG-CAP.

   If a Whois++ DAG-CAP receives several different response codes from
   DAG-SAPs it should try to represent them all in the response to the
   end-user client.

   The proposed mapping between DAG/IP response codes and Whois++
   response codes are given in Appendix D.



(page 42 continued on part 3)

Next RFC Part