US8737210B2 - Load balancing SCTP associations using VTAG mediation - Google Patents

Load balancing SCTP associations using VTAG mediation Download PDF

Info

Publication number
US8737210B2
US8737210B2 US13/044,128 US201113044128A US8737210B2 US 8737210 B2 US8737210 B2 US 8737210B2 US 201113044128 A US201113044128 A US 201113044128A US 8737210 B2 US8737210 B2 US 8737210B2
Authority
US
United States
Prior art keywords
sctp
end server
distribution key
tag
client
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US13/044,128
Other versions
US20120230187A1 (en
Inventor
Richard Tremblay
Abdallah CHATILLA
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telefonaktiebolaget LM Ericsson AB
Original Assignee
Telefonaktiebolaget LM Ericsson AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonaktiebolaget LM Ericsson AB filed Critical Telefonaktiebolaget LM Ericsson AB
Priority to US13/044,128 priority Critical patent/US8737210B2/en
Priority to PCT/IB2012/051096 priority patent/WO2012120473A1/en
Priority to CN201280022601.1A priority patent/CN103814565A/en
Priority to EP12711006.2A priority patent/EP2684339A1/en
Publication of US20120230187A1 publication Critical patent/US20120230187A1/en
Assigned to TELEFONAKTIEBOLAGET L M ERICSSON (PUBL) reassignment TELEFONAKTIEBOLAGET L M ERICSSON (PUBL) ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHATILA, ABDALLAH, TREMBLAY, RICHARD
Application granted granted Critical
Publication of US8737210B2 publication Critical patent/US8737210B2/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1027—Persistence of sessions during load balancing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004—Server selection for load balancing
    • H04L67/1023—Server selection for load balancing based on a hash applied to IP addresses or costs

Definitions

  • the present invention relates generally to load balancing a series of servers and more specifically to load balancing a series of servers based on a verification tag associated with a Stream Control Transport Protocol (SCTP).
  • SCTP Stream Control Transport Protocol
  • SCTP Stream Control Transmission Protocol
  • TCP Transmission Control Protocol
  • a load-balancing system for multiple servers is desired that provides the features of SCTP but with one or more of: 1) no modifications to the SCTP protocol; 2) minimize the amount of SCTP chunk inspection; 3) minimize association state storing; 4) minimize SCTP checksum recalculation; 5) no modifications to the IP header; 6) support the SCTP multi-homing feature; 7) transparent to users of the socket Application Programming Interface (API); and 8) no modifications to the server IP communications stack.
  • a number of attempts, based on a Network Address Translation (NAT) scheme, to provide a solution have been attempted but these solutions typically do not meet some or all of the characteristics specified above.
  • NAT Network Address Translation
  • Systems and methods address the market needs described above by providing an intermediate front-end server to route SCTP communications between clients requesting a service and back-end servers providing the service.
  • the front-end server and a series of back-end servers share a Virtual Internet Protocol (VIP) address and SCTP port numbers allowing the clients to access the service without knowledge of the specific back-end server providing the service.
  • VIP Virtual Internet Protocol
  • the back-end servers operate independently and are not aware that other back-end servers exist or that a front-end server is acting as an intermediary.
  • the client is unaware of the presence of the front-end server and believes the SCTP communication interaction is directly with the back-end server.
  • a method for providing SCTP communication traffic distribution based on SCTP verification tags.
  • a request is received at a front-end server to initiate an SCTP association between a client and a back-end server.
  • the front-end server generates a distribution key, if clashing Verification-Tags are detected, for mediating SCTP communication packets between the client and the back-end server.
  • the front-end server transparently routes SCTP packets between the client and the back-end server.
  • a system for providing a transparent intermediary SCTP communication routing front-end server includes the following components.
  • the first exemplary system embodiment component is an initialization component for establishing an SCTP association between a client and a back-end server and creating a distribution key for the SCTP association based on SCTP information provided by the client and the back-end server.
  • the next exemplary system embodiment component is an engine component for routing the SCTP communication traffic between the client and the back-end server based on the SCTP distribution key.
  • the next exemplary system embodiment component is a storage component for storing data associated with routing the SCTP communications between the client and the back-end server.
  • the front-end server if a clash develops between the distribution key created for one SCTP association and the distribution key created for another SCTP association managed by the front-end server, then the front-end server generates a new distribution key by replacing a portion of the information provided by the back-end server with information provided by the front-end server. Continuing with the exemplary embodiment, the front-end server then manages this matched pair of distribution keys for the clashing distribution key association in verification tag translation table.
  • FIG. 1 depicts a system for a load-balancing front-end node to establish and route an SCTP connection between a client and a back-end serving node based on a back-end serving node generated SCTP verification tag;
  • FIG. 2 depicts a front-end server for establishing and directing a connection between a client and a back-end serving node based on a back-end serving node generated SCTP verification tag wherein the front-end server is facilitated by an initialization component, an engine component and a storage component;
  • FIG. 3 depicts an initialization component of a front-end component of a system for a load-balancing front-end node to establish and direct a connection between a client and a back-end serving node based on a back-end serving node generated SCTP verification tag wherein the initialization component is facilitated by a client initializing component and a back-end server initializing component;
  • FIG. 4 is a flowchart depicting an SCTP association requests and responses between a client and a back-end server through a load-balancing front-end server with the client initiating the communication;
  • FIG. 5 is a flowchart depicting an SCTP association requests and responses between a client and a back-end server through a load-balancing front-end server with the back-end server initiating the communication;
  • FIG. 6 is a flowchart depicting after SCTP association communications from a client to a back-end server through a load-balancing front-end server;
  • FIG. 7 a flowchart depicting a method for establishing an SCTP association between a client and one of a plurality of back-end servers through a load-balancing front-end server while preventing the possibility of clashing distribution based on SCTP verification tags generated by the back-end servers.
  • FIG. 8 depicts an exemplary computing device for implementing a system for a load-balancing front-end node to establish and route an SCTP connection between a client and a back-end serving node based on a back-end serving node generated SCTP verification tag
  • FIG. 1 a diagram of an exemplary embodiment of a load-balancing SCTP association system 100 for providing communication distribution based on verification tag mediation is illustrated.
  • the exemplary embodiment of the load-balancing SCTP association system 100 includes but is not limited to an exemplary client 102 , an exemplary network 104 , an exemplary front-end node 106 and three exemplary back-end servers 108 . It should be noted in this exemplary embodiment that the back-end servers 108 can be any number of back-end servers 108 operating independently.
  • the client 102 is any device capable of requesting a service from a front-end server 106 communicatively connected to the client 102 across a network 104 .
  • the client 102 includes but is not limited to a personal computer running a web browser and accessing a web page located at a website on the internet.
  • the client 102 is configured to communicate to the front-end server 106 with the Stream Control Transport Protocol (SCTP) for connection-oriented support.
  • SCTP Stream Control Transport Protocol
  • the client 102 is a telephone connected to a Voice over Internet Protocol (VoIP) device to communicate across a network 104 such as the internet to a front-end server for voice communications.
  • VoIP Voice over Internet Protocol
  • the network 104 provides a communications link between the client 102 and the front-end server 106 .
  • the network 104 can be the internet.
  • a front-end node 106 provides the capability to transparently route communications between a client 102 and one of a series of back-end servers 108 by using the SCTP verification tag as a distribution key.
  • the series of back-end servers 108 provide the application services desired by the client 102 . It should be noted that although a single client 102 is illustrated, a plurality of clients 102 can be connected to the series of back-end servers 108 .
  • the back-end servers 108 are unaware of each other and operate independently with their connected clients 102 . It should also be noted that the back-end servers 108 and the client(s) 102 are unaware of the front-end server 106 , the front-end server is transparent to the connection between the client(s) 102 and the back-end servers 108 and routes communications between the client(s) 102 and the back-end servers 108 based on the SCTP verification tags created by the back-end server 108 for the SCTP association.
  • FIG. 2 another exemplary embodiment 200 is depicted as a portion of exemplary embodiment 100 .
  • Exemplary embodiment 200 depicts a front-end server 106 including an initialization component 202 , an engine component 204 and a storage component 206 .
  • the initialization component 202 can provide the capability to facilitate the creation of a non-clashing SCTP connection from either a client 102 or a back-end server 108 .
  • the initialization component 202 can generate a distribution key based on a combination of the client 102 provided SCTP port number, the back-end server 108 provided SCTP port number and the back-end server 108 provided SCTP Initiate-Tag.
  • the front-end server 106 uses the distribution key to route communications between a client 102 and a back-end server 108 and guaranty that all communications received at the front-end server 106 are delivered to the appropriate end-point.
  • the initialization component 202 creates and maintains a verification tag translation table to prevent any clash between distribution keys.
  • a clash would develop if two client 102 /back-end server 108 pairs provided port numbers and an Initiate-Tag that combined to form identical distribution tags.
  • the front-end server 106 is initiating an SCTP association between a client 102 and a back-end server 108 the front-end server creates the distribution key based on the client 102 /back-end server 108 port numbers and the Initiate-Tag provided by the back-end server 108 .
  • the front-end server 106 looks in the verification tag translation table for an identical distribution key and if none is found then the SCTP association as initialized can continue with the front-end server 106 correctly routing communications between the client 102 and the back-end server 108 based on the distribution key.
  • the front-end server 106 finds a match of the distribution key in the verification tag translation table then the front-end server 106 generates a new Initiate-Tag value and creates a new non-conflicting distribution key.
  • the front-end server 106 creates a new entry in the verification tag translation table to hold the distribution key pair and the association initialization continues with the front-end server 106 correctly routing communications between the client 102 and the back-end server 108 based on the distribution key pair maintained in the verification tag translation table by the front-end server 106 .
  • the engine component 204 provides the ability to distribute communications between a client 102 and a back-end server 108 after completion of the SCTP association initialization.
  • the front-end server 106 receives a SCTP communication from a client 102 directed to one of the back-end servers 108 sharing a virtual internet protocol (VIP) address with the front-end server 106 .
  • VIP virtual internet protocol
  • the engine component 204 of the front-end server 106 attempts to find the distribution key of the SCTP communication in the verification tag translation table and if the distribution key is not found in the verification tag translation table then the engine component 204 of the front-end server 106 forwards the SCTP communication to the back-end server 108 specified by the distribution key. Further in the exemplary embodiment, if the distribution key is found in the verification tag translation table then the engine component 204 of the front-end server 106 substitutes the distribution key in the communication with the associated distribution key in the verification tag translation table and recalculates the checksum, if required, for the communication and forwards the communication to the back-end server 108 specified by the replacement distribution key.
  • a storage component 206 provides the ability to store data associated with maintaining SCTP associations between a client 102 and a back-end server 108 . Further in the exemplary environment, the storage component 206 comprises a verification tag translation table and a count of the number of entries in the verification tag translation table. The verification tag translation table counter in the exemplary environment storage component 206 can be used to determine if there is any need to inspect the verification tag translation table, as long as the count is zero, there have not been any clashes in distribution key generation and the communications from any clients 102 to any back-end servers 108 can be forwarded without a search of the verification tag translation table.
  • FIG. 3 another exemplary embodiment 300 is depicted.
  • a portion of the exemplary embodiment 300 depicts a client initialization component 302 and a back-end server initialization component 304 .
  • the client initialization component 302 provides the capability to manage an SCTP association initiated by a client 102 .
  • the client initializing component 302 determines if the Initiate-Tag provided by the back-end server 108 would create a clashing distribution key with another SCTP association.
  • the client initializing component 302 would replace the Initiate-Tag generated by the back-end server 108 with a non-clashing Initiate-Tag generated by the client initializing component 302 , place the non-clashing Initiate-Tag in the INIT-ACK chunk and recalculate and replace the checksum in the SCTP common header.
  • the back-end server initializing component provides the capability to manage an SCTP association initiated by a back-end server 108 .
  • the back-end server initializing component 304 determines if the Initiate-Tag provided by the back-end server 108 would create a clashing distribution key with another SCTP association.
  • the back-end server initializing component 304 would replace the Initiate-Tag generated by the back-end server 108 with a non-clashing Initiate-Tag generated by the back-end server initializing component 304 , place the non-clashing Initiate-Tag in the INIT chunk and recalculate and replace the checksum in the SCTP common header.
  • the exemplary embodiment 400 depicts the signaling flow for a client 102 initiating an SCTP association with a back-end server 108 through a front-end server 106 .
  • the front-end server 106 and one or more back-end servers 108 share a virtual internet protocol (IP) address and the back-end servers 108 operate independently of each other.
  • IP internet protocol
  • the operation of the front-end server 106 is transparent to both the client 102 and the back-end server 108 involved in the SCTP association.
  • the client 102 sends an SCTP INIT chunk towards the virtual IP address shared by the front-end server 106 and the series of back-end servers 108 .
  • the front-end server 106 receives the SCTP INIT chunk and makes a determination based on distribution policies which back-end server 108 will receive the SCTP INIT chunk.
  • the front-end server 106 forwards the SCTP INIT chunk to the selected back-end server 108 .
  • the back-end server 108 processes the SCTP INIT chunk by generating an SCTP INIT-ACK chunk including an Initiate-Tag and the SCTP port number used by the back-end server 108 and at 406 , sends the INIT-ACK chunk towards the client 102 .
  • the front-end server 106 receives the SCTP INIT-ACK chunk and inspects the contents of the INIT-ACK chunk to create a distribution key to manage the communications between the initiating client 102 and the selected back-end server 108 .
  • the exemplary embodiment continues with the front-end server 106 combining the client 102 SCTP port number with the Initiate-Tag and the back-end server 108 SCTP port number to create a distribution key for the SCTP association.
  • the front-end server 106 checks the verification tag translation table to confirm that the newly created distribution key is not already in use by another SCTP association managed by the front-end server 106 . In the exemplary embodiment, if the distribution key is found in the verification tag translation table then the front-end server 106 generates a new Initiate-Tag and creates a non-clashing distribution key.
  • the front-end server creates a new entry in the verification tag translation table for the client 102 and back-end server 108 generated Initiate-Tags and stores the values in the verification tag translation table.
  • the front-end server 106 updates the SCTP INIT-ACK chunk with the new Initiate-Tag and a recalculated checksum and, at step 408 , forwards the updated SCTP INIT-ACK chunk to the client 102 . It should be noted in the exemplary embodiment that if the front-end server 106 does not detect a clash of distribution keys then the front-end server 106 does not create an entry in the verification tag translation table for the SCTP association.
  • the client 102 sends a COOKIE-ECHO chunk towards the back-end server 108 and the intermediate front-end server 106 inspects the COOKIE-ECHO chunk to determine if the distribution key is a match with any of the distribution keys stored in the verification tag translation table.
  • the front-end server 106 replaces the Verification-Tag in the COOKIE-ECHO chunk with the Initiate tag from the verification tag translation table, replaces the checksum with a checksum recalculated based on the replaced Verification-Tag and, at step 412 , forwards the COOKIE-ECHO chunk to the back-end server 108 .
  • the back-end server 108 sends a COOKIE-ACK chunk towards the client 102 and at step 416 the front-end server 106 transparently forwards the COOKIE-ACK chunk towards the client 102 .
  • the exemplary embodiment 500 depicts the signaling flow for a back-end server 108 initiating an SCTP association with a client 102 through a front-end server 106 .
  • the front-end server 106 and one or more back-end servers 108 share a virtual internet protocol (IP) address and the back-end servers 108 operate independently of each other.
  • IP internet protocol
  • the operation of the front-end server 106 is transparent to both the client 102 and the back-end server 108 involved in the SCTP association.
  • the back-end server 108 generates an Initiate-Tag and sends the Initiate-Tag, at step 502 , in an SCTP INIT chunk towards the client 102 transparently through the front-end server 106 .
  • the front-end server 106 receives the SCTP INIT chunk from the back-end server 108 and transparently inspects the contents of the INIT chunk to create a distribution key to manage the communications between the destination client 102 and the initiating back-end server 108 .
  • the exemplary embodiment continues with the front-end server 106 combining the client SCTP port number with the back-end server 108 generated Initiate-Tag and the back-end server 108 SCTP port number to create a distribution key for the SCTP association.
  • the front-end server 106 checks the verification tag translation table to confirm that the newly created distribution key is not already in use by another SCTP association managed by the front-end server 106 . In the exemplary embodiment, if the distribution key is found in the verification tag translation table then the front-end server 106 generates a new Initiate-Tag to replace the back-end server 108 generated Initiate-Tag and creates a non-clashing distribution key. Next in the exemplary embodiment, the front-end server creates a new entry in the verification tag translation table for the client 102 and back-end server 108 generated Initiate-Tag and SCTP port numbers and stores the values in the verification tag translation table.
  • the front-end server 106 forwards the SCTP INIT chunk to the client 102 and the client 102 processes the SCTP INIT chunk by generating an SCTP INIT-ACK chunk including a client generated Initiate-Tag and a cookie associated with the client and, at step 506 , sends the INIT-ACK chunk towards the back-end server 108 through the front-end server 106 .
  • the front-end server 106 receives the SCTP INIT-ACK chunk from the client 102 and transparently inspects the contents of the SCTP packet common header to retrieve the distribution key used to distribute the SCTP INIT-ACK to the appropriate back-end server 108 .
  • the front-end server 106 checks the verification tag translation table to determine if the distribution key is in the verification tag translation table.
  • the front-end server 106 replaces the Verification-Tag in the SCTP common header of the INIT-ACK message with the associated back-end server 108 Initiate-Tag from the verification tag translation table and updates the checksum before forwarding the SCTP INIT-ACK to the appropriate back-end server 108 at step 508 . It should be noted in the exemplary embodiment that if the front-end server 106 does not detect a clash of distribution keys then the front-end server 106 simply forwards the SCTP INIT-ACK to the appropriate back-end server 108 based on the Verification-Tag retrieved from the SCTP common header and the back-end server establishes an SCTP association with the client.
  • the back-end server 108 sends a COOKIE-ECHO chunk towards the client 102 through the front-end server 106 and the front-end server 106 transparently forwards, at step 512 , the COOKIE-ECHO to the client 102 and the client establishes an SCTP association with the back-end server 108 .
  • the client 102 sends a COOKIE-ACK chunk towards the back-end server 108 and at step 516 the front-end server 106 determines if a distribution key exists for this SCTP association and accordingly if an exchange of Verification-Tags is required.
  • the exemplary embodiment continues with the front-end server 106 transparently, with regard to the client 102 and the back-end server 108 , forwarding the COOKIE-ACK chunk towards the back-end server 108 .
  • the exemplary embodiment 600 depicts the signaling flow for a client 102 communicating through a front-end server 106 to a back-end server 108 using an established SCTP association. It should be noted in the exemplary embodiment that the front-end server 106 and one or more back-end servers 108 share a virtual internet protocol (IP) address and the back-end servers 108 operate independently of each other. It should be further noted in the exemplary embodiment that the operation of the front-end server 106 is transparent to both the client 102 and the back-end server 108 involved in the SCTP association.
  • IP internet protocol
  • a client 102 sends, at step 602 , an SCTP packet through the front end server 106 towards a back-end server 108 .
  • the front-end server 106 receives the SCTP packet from the client 102 and transparently inspects the contents of the SCTP packet to retrieve the distribution key used to distribute the SCTP packet to the appropriate back-end server 108 .
  • the front-end server 106 checks the verification tag translation table to determine if the distribution key is in the verification tag translation table.
  • the front-end server 106 replaces the Verification-Tag in the SCTP packet common header with the associated back-end server 108 Initiate-Tag from the verification tag translation table and updates the checksum before forwarding the SCTP packet to the appropriate back-end server 108 at step 604 . It should be noted in the exemplary embodiment that if the front-end server 106 does not detect a clash of distribution keys then the front-end server 106 forwards the SCTP packet to the appropriate back-end server 108 based on the Verification-Tag retrieved from the SCTP packet common header.
  • an exemplary method embodiment 700 for providing SCTP communication traffic distribution based on SCTP verification tags can receive a request to initiate an SCTP association between a client 102 and a back-end server 108 .
  • the initiation request can come from the client 102 or the back-end server.
  • method 700 operates on a front-end server 106 that is transparent to both the client 102 and the back-end server 108 .
  • the front-end server 106 can generate a distribution key for routing SCTP communication packets between the client 102 and the back-end server 108 based on SCTP parameters collected from the client and the back-end server 108 .
  • the front-end server 106 can use the SCTP port number from the client 102 , the SCTP port number from the back-end server 108 and the SCTP Verification-Tag from the back-end server to generate a distribution tag that for this SCTP association.
  • the method 700 transparently routes SCTP packets between the client 102 and the back-end server 108 .
  • the exemplary method embodiment 700 retrieves the distribution key from the SCTP packet and extracts the SCTP Verification-Tag from the distribution key.
  • the SCTP Verification-Tag represents the identity of the client 102 or back-end server 108 destination connected to the SCTP association for this SCTP communication.
  • the exemplary method embodiment 700 then sends this SCTP packet towards the identified client 102 or back-end server 108 .
  • FIG. 8 illustrates an example of a suitable computing system environment 800 in which the claimed subject matter can be implemented, although as made clear above, the computing system environment 800 is only one example of a suitable computing environment for an exemplary embodiment and is not intended to suggest any limitation as to the scope of use or functionality of the claimed subject matter. Further, the computing environment 800 is not intended to suggest any dependency or requirement relating to the claimed subject matter and any one or combination of components illustrated in the example computing environment 800 .
  • an example of a device for implementing the previously described innovation includes a general purpose computing device in the form of a computer 810 .
  • Components of computer 810 can include, but are not limited to, a processing unit 820 , a system memory 830 , and a system bus 890 that couples various system components including the system memory to the processing unit 820 .
  • the system bus 890 can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
  • Computer 810 can include a variety of computer readable media.
  • Computer readable media can be any available media that can be accessed by computer 810 .
  • Computer readable media can comprise computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile as well as removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CDROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 810 .
  • Communication media can embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and can include any suitable information delivery media.
  • the system memory 830 can include computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) and/or random access memory (RAM).
  • ROM read only memory
  • RAM random access memory
  • a basic input/output system (BIOS) containing the basic routines that help to transfer information between elements within computer 810 , such as during start-up, can be stored in memory 830 .
  • Memory 830 can also contain data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 820 .
  • memory 830 can also include an operating system, application programs, other program modules, and program data.
  • the computer 810 can also include other removable/non-removable and volatile/nonvolatile computer storage media.
  • computer 810 can include a hard disk drive that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive that reads from or writes to a removable, nonvolatile magnetic disk, and/or an optical disk drive that reads from or writes to a removable, nonvolatile optical disk, such as a CD-ROM or other optical media.
  • Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM and the like.
  • a hard disk drive can be connected to the system bus 890 through a non-removable memory interface such as an interface, and a magnetic disk drive or optical disk drive can be connected to the system bus 890 by a removable memory interface, such as an interface.
  • a user can enter commands and information into the computer 810 through input devices such as a keyboard or a pointing device such as a mouse, trackball, touch pad, and/or other pointing device.
  • Other input devices can include a microphone, joystick, game pad, satellite dish, scanner, or similar devices.
  • These and/or other input devices can be connected to the processing unit 820 through user input 840 and associated interface(s) that are coupled to the system bus 890 , but can be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB).
  • USB universal serial bus
  • a graphics subsystem can also be connected to the system bus 890 .
  • a monitor or other type of display device can be connected to the system bus 890 through an interface, such as output interface 850 , which can in turn communicate with video memory.
  • computers can also include other peripheral output devices, such as speakers and/or printing devices, which can also be connected through output interface 850 .
  • the computer 810 can operate in a networked or distributed environment using logical connections to one or more other remote computers, such as remote server 870 , which can in turn have media capabilities different from device 810 .
  • the remote server 870 can be a personal computer, a server, a router, a network PC, a peer device or other common network node, and/or any other remote media consumption or transmission device, and can include any or all of the elements described above relative to the computer 810 .
  • the logical connections depicted in FIG. 8 include a network 880 , such as a local area network (LAN) or a wide area network (WAN), but can also include other networks/buses.
  • LAN local area network
  • WAN wide area network
  • the computer 810 When used in a LAN networking environment, the computer 810 is connected to the LAN 880 through a network interface or adapter. When used in a WAN networking environment, the computer 810 can include a communications component, such as a modem, or other means for establishing communications over a WAN, such as the Internet.
  • a communications component such as a modem, which can be internal or external, can be connected to the system bus 890 through the user input interface at input 840 and/or other appropriate mechanism.
  • program modules depicted relative to the computer 810 can be stored in a remote memory storage device. It should be noted that the network connections shown and described are exemplary and other means of establishing a communications link between the computers can be used.
  • a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program and a computing device.
  • a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program and a computing device.
  • both an application running on a computing device and the computing device can be components.
  • One or more components can reside within a process and/or thread of execution and a component can be localized on one computing device and/or distributed between two or more computing devices, and/or communicatively connected modules. Further, it should be noted that as used in this application, terms such as “system user,” “user,” and similar terms are intended to refer to the person operating the computing device referenced above.
  • the term to “infer” or “inference” refer generally to the process of reasoning about or inferring states of the system, environment, user, and/or intent from a set of observations captured from events and/or data. Captured events and data can include user data, device data, environment data, behavior data, application data, implicit and explicit data, etc. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The inference can be probabilistic in that the computation of a probability distribution over states of interest based on a consideration of data and events. Inference can also refer to techniques employed for composing higher-level events from a set of events and/or data. Such inference results in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources.

Abstract

Presented is a system and methods for routing Stream Control Transmission Protocol (SCTP) between a client and a back-end server by a front-end server operating transparently between the client and the back-end server. The front-end server uses a distribution key created by the front-end server to route the SCTP packets to the proper destination. The distribution key is comprised of a client SCTP port number, a back-end server SCTP port number and a back-end server Initiate-Tag. The front-end server arbitrates clashes between distribution keys of different back-end servers by replacing the back-end server generated SCTP Initiate-Tag with a front-end server generated SCTP Initiate-Tag.

Description

TECHNICAL FIELD
The present invention relates generally to load balancing a series of servers and more specifically to load balancing a series of servers based on a verification tag associated with a Stream Control Transport Protocol (SCTP).
BACKGROUND
As the popularity of the internet and the functionality of websites continue to grow, many websites require multiple servers to handle the load of communications traffic directed toward their pages. In another use of the internet, Voice over Internet Protocol service has grown to a volume where many servers are required to handle the demand for a given service provider. As the requirement for multi-server systems evolves, a need arises for the ability to balance the load generated for the service across the number of deployed servers providing the service.
Further, the desire to handle the signaling of telecommunications over Internet Protocol (IP) and the growth in complexity of websites with regard to providing a rich multimedia experience combined with reliable and responsive communications has led to the development of communication protocols such as Stream Control Transmission Protocol (SCTP). SCTP provides a connection-oriented protocol, similar to Transmission Control Protocol (TCP), on top of the connectionless IP and includes the additional features of multi-homing and multi-streaming that are not available with TCP. These additional features allow a more efficient communication between a multitude of clients and servers.
A load-balancing system for multiple servers is desired that provides the features of SCTP but with one or more of: 1) no modifications to the SCTP protocol; 2) minimize the amount of SCTP chunk inspection; 3) minimize association state storing; 4) minimize SCTP checksum recalculation; 5) no modifications to the IP header; 6) support the SCTP multi-homing feature; 7) transparent to users of the socket Application Programming Interface (API); and 8) no modifications to the server IP communications stack. A number of attempts, based on a Network Address Translation (NAT) scheme, to provide a solution have been attempted but these solutions typically do not meet some or all of the characteristics specified above.
Consequently, market pressure is building for a load-balancing capable system which would meet the characteristics specified above and would also allow, among other things, the ability to scale the system capacity as required without interference with the currently operating servers or the applications and associations running on the operating servers.
SUMMARY
Systems and methods address the market needs described above by providing an intermediate front-end server to route SCTP communications between clients requesting a service and back-end servers providing the service. The front-end server and a series of back-end servers share a Virtual Internet Protocol (VIP) address and SCTP port numbers allowing the clients to access the service without knowledge of the specific back-end server providing the service. In fact, according to an embodiment the back-end servers operate independently and are not aware that other back-end servers exist or that a front-end server is acting as an intermediary. In a similar fashion, the client is unaware of the presence of the front-end server and believes the SCTP communication interaction is directly with the back-end server.
In one exemplary embodiment, a method is illustrated for providing SCTP communication traffic distribution based on SCTP verification tags. In a first exemplary embodiment step, a request is received at a front-end server to initiate an SCTP association between a client and a back-end server. In the next exemplary embodiment step, the front-end server generates a distribution key, if clashing Verification-Tags are detected, for mediating SCTP communication packets between the client and the back-end server. In the next exemplary embodiment step, the front-end server transparently routes SCTP packets between the client and the back-end server.
In another exemplary embodiment, a system for providing a transparent intermediary SCTP communication routing front-end server is presented. The exemplary system embodiment includes the following components. The first exemplary system embodiment component is an initialization component for establishing an SCTP association between a client and a back-end server and creating a distribution key for the SCTP association based on SCTP information provided by the client and the back-end server. The next exemplary system embodiment component is an engine component for routing the SCTP communication traffic between the client and the back-end server based on the SCTP distribution key. Continuing, the next exemplary system embodiment component is a storage component for storing data associated with routing the SCTP communications between the client and the back-end server.
Further, in another exemplary embodiment, if a clash develops between the distribution key created for one SCTP association and the distribution key created for another SCTP association managed by the front-end server, then the front-end server generates a new distribution key by replacing a portion of the information provided by the back-end server with information provided by the front-end server. Continuing with the exemplary embodiment, the front-end server then manages this matched pair of distribution keys for the clashing distribution key association in verification tag translation table.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings illustrate exemplary embodiments, wherein:
FIG. 1 depicts a system for a load-balancing front-end node to establish and route an SCTP connection between a client and a back-end serving node based on a back-end serving node generated SCTP verification tag;
FIG. 2 depicts a front-end server for establishing and directing a connection between a client and a back-end serving node based on a back-end serving node generated SCTP verification tag wherein the front-end server is facilitated by an initialization component, an engine component and a storage component;
FIG. 3 depicts an initialization component of a front-end component of a system for a load-balancing front-end node to establish and direct a connection between a client and a back-end serving node based on a back-end serving node generated SCTP verification tag wherein the initialization component is facilitated by a client initializing component and a back-end server initializing component;
FIG. 4 is a flowchart depicting an SCTP association requests and responses between a client and a back-end server through a load-balancing front-end server with the client initiating the communication;
FIG. 5 is a flowchart depicting an SCTP association requests and responses between a client and a back-end server through a load-balancing front-end server with the back-end server initiating the communication;
FIG. 6 is a flowchart depicting after SCTP association communications from a client to a back-end server through a load-balancing front-end server;
FIG. 7 a flowchart depicting a method for establishing an SCTP association between a client and one of a plurality of back-end servers through a load-balancing front-end server while preventing the possibility of clashing distribution based on SCTP verification tags generated by the back-end servers.
FIG. 8 depicts an exemplary computing device for implementing a system for a load-balancing front-end node to establish and route an SCTP connection between a client and a back-end serving node based on a back-end serving node generated SCTP verification tag
 DETAILED DESCRIPTION
The following detailed description of the exemplary embodiments refers to the accompanying drawings. The same reference numbers in different drawings identify the same or similar elements. Also, the following detailed description does not limit the invention. Instead, the scope of the invention is defined by the appended claims.
Looking first to FIG. 1, a diagram of an exemplary embodiment of a load-balancing SCTP association system 100 for providing communication distribution based on verification tag mediation is illustrated. The exemplary embodiment of the load-balancing SCTP association system 100 includes but is not limited to an exemplary client 102, an exemplary network 104, an exemplary front-end node 106 and three exemplary back-end servers 108. It should be noted in this exemplary embodiment that the back-end servers 108 can be any number of back-end servers 108 operating independently.
In one aspect of the exemplary embodiment, the client 102 is any device capable of requesting a service from a front-end server 106 communicatively connected to the client 102 across a network 104. In one example of the exemplary embodiment the client 102 includes but is not limited to a personal computer running a web browser and accessing a web page located at a website on the internet. In another aspect of the exemplary embodiment, the client 102 is configured to communicate to the front-end server 106 with the Stream Control Transport Protocol (SCTP) for connection-oriented support. Further in the exemplary embodiment, the client 102 is a telephone connected to a Voice over Internet Protocol (VoIP) device to communicate across a network 104 such as the internet to a front-end server for voice communications.
In another aspect of the exemplary embodiment, the network 104 provides a communications link between the client 102 and the front-end server 106. In one configuration of the exemplary embodiment, the network 104 can be the internet. Continuing with the exemplary embodiment, a front-end node 106 provides the capability to transparently route communications between a client 102 and one of a series of back-end servers 108 by using the SCTP verification tag as a distribution key. In a further aspect of the exemplary embodiment, the series of back-end servers 108 provide the application services desired by the client 102. It should be noted that although a single client 102 is illustrated, a plurality of clients 102 can be connected to the series of back-end servers 108. In a further aspect of the exemplary embodiment, the back-end servers 108 are unaware of each other and operate independently with their connected clients 102. It should also be noted that the back-end servers 108 and the client(s) 102 are unaware of the front-end server 106, the front-end server is transparent to the connection between the client(s) 102 and the back-end servers 108 and routes communications between the client(s) 102 and the back-end servers 108 based on the SCTP verification tags created by the back-end server 108 for the SCTP association.
Looking now to FIG. 2, another exemplary embodiment 200 is depicted as a portion of exemplary embodiment 100. Exemplary embodiment 200 depicts a front-end server 106 including an initialization component 202, an engine component 204 and a storage component 206. In one aspect of the exemplary embodiment 200 the initialization component 202 can provide the capability to facilitate the creation of a non-clashing SCTP connection from either a client 102 or a back-end server 108.
In another aspect of the exemplary embodiment 200, the initialization component 202 can generate a distribution key based on a combination of the client 102 provided SCTP port number, the back-end server 108 provided SCTP port number and the back-end server 108 provided SCTP Initiate-Tag. Continuing with the exemplary embodiment, the front-end server 106 uses the distribution key to route communications between a client 102 and a back-end server 108 and guaranty that all communications received at the front-end server 106 are delivered to the appropriate end-point.
In another aspect of the exemplary embodiment 200, the initialization component 202 creates and maintains a verification tag translation table to prevent any clash between distribution keys. In this exemplary embodiment, a clash would develop if two client 102/back-end server 108 pairs provided port numbers and an Initiate-Tag that combined to form identical distribution tags. Continuing with the exemplary embodiment, as the front-end server 106 is initiating an SCTP association between a client 102 and a back-end server 108 the front-end server creates the distribution key based on the client 102/back-end server 108 port numbers and the Initiate-Tag provided by the back-end server 108. Next in the exemplary embodiment, the front-end server 106 looks in the verification tag translation table for an identical distribution key and if none is found then the SCTP association as initialized can continue with the front-end server 106 correctly routing communications between the client 102 and the back-end server 108 based on the distribution key.
Further in the exemplary embodiment, if the front-end server 106 finds a match of the distribution key in the verification tag translation table then the front-end server 106 generates a new Initiate-Tag value and creates a new non-conflicting distribution key. Next in the exemplary embodiment, the front-end server 106 creates a new entry in the verification tag translation table to hold the distribution key pair and the association initialization continues with the front-end server 106 correctly routing communications between the client 102 and the back-end server 108 based on the distribution key pair maintained in the verification tag translation table by the front-end server 106.
In another aspect of the exemplary embodiment, the engine component 204 provides the ability to distribute communications between a client 102 and a back-end server 108 after completion of the SCTP association initialization. In one aspect of the exemplary environment, the front-end server 106 receives a SCTP communication from a client 102 directed to one of the back-end servers 108 sharing a virtual internet protocol (VIP) address with the front-end server 106. Continuing with the exemplary embodiment, the engine component 204 of the front-end server 106 attempts to find the distribution key of the SCTP communication in the verification tag translation table and if the distribution key is not found in the verification tag translation table then the engine component 204 of the front-end server 106 forwards the SCTP communication to the back-end server 108 specified by the distribution key. Further in the exemplary embodiment, if the distribution key is found in the verification tag translation table then the engine component 204 of the front-end server 106 substitutes the distribution key in the communication with the associated distribution key in the verification tag translation table and recalculates the checksum, if required, for the communication and forwards the communication to the back-end server 108 specified by the replacement distribution key.
Continuing with another aspect of the exemplary embodiment, a storage component 206 provides the ability to store data associated with maintaining SCTP associations between a client 102 and a back-end server 108. Further in the exemplary environment, the storage component 206 comprises a verification tag translation table and a count of the number of entries in the verification tag translation table. The verification tag translation table counter in the exemplary environment storage component 206 can be used to determine if there is any need to inspect the verification tag translation table, as long as the count is zero, there have not been any clashes in distribution key generation and the communications from any clients 102 to any back-end servers 108 can be forwarded without a search of the verification tag translation table.
Turning now to FIG. 3, another exemplary embodiment 300 is depicted. A portion of the exemplary embodiment 300 depicts a client initialization component 302 and a back-end server initialization component 304. In one aspect of the exemplary embodiment 300, the client initialization component 302 provides the capability to manage an SCTP association initiated by a client 102. In the exemplary embodiment, the client initializing component 302 determines if the Initiate-Tag provided by the back-end server 108 would create a clashing distribution key with another SCTP association. Continuing with the exemplary embodiment, if a clashing distribution key is detected then the client initializing component 302 would replace the Initiate-Tag generated by the back-end server 108 with a non-clashing Initiate-Tag generated by the client initializing component 302, place the non-clashing Initiate-Tag in the INIT-ACK chunk and recalculate and replace the checksum in the SCTP common header.
Continuing with the exemplary embodiment, the back-end server initializing component provides the capability to manage an SCTP association initiated by a back-end server 108. In the exemplary embodiment, the back-end server initializing component 304 determines if the Initiate-Tag provided by the back-end server 108 would create a clashing distribution key with another SCTP association. Continuing with the exemplary embodiment, if a clashing distribution key is detected then the back-end server initializing component 304 would replace the Initiate-Tag generated by the back-end server 108 with a non-clashing Initiate-Tag generated by the back-end server initializing component 304, place the non-clashing Initiate-Tag in the INIT chunk and recalculate and replace the checksum in the SCTP common header.
Turning now to FIG. 4, illustrated is an exemplary embodiment 400. The exemplary embodiment 400 depicts the signaling flow for a client 102 initiating an SCTP association with a back-end server 108 through a front-end server 106. It should be noted in the exemplary embodiment that the front-end server 106 and one or more back-end servers 108 share a virtual internet protocol (IP) address and the back-end servers 108 operate independently of each other. It should be further noted in the exemplary embodiment that the operation of the front-end server 106 is transparent to both the client 102 and the back-end server 108 involved in the SCTP association.
First, at exemplary embodiment step 402, the client 102 sends an SCTP INIT chunk towards the virtual IP address shared by the front-end server 106 and the series of back-end servers 108. In the exemplary embodiment, the front-end server 106 receives the SCTP INIT chunk and makes a determination based on distribution policies which back-end server 108 will receive the SCTP INIT chunk. Continuing at step 404 with the exemplary embodiment, the front-end server 106 forwards the SCTP INIT chunk to the selected back-end server 108. Continuing with the exemplary embodiment, the back-end server 108 processes the SCTP INIT chunk by generating an SCTP INIT-ACK chunk including an Initiate-Tag and the SCTP port number used by the back-end server 108 and at 406, sends the INIT-ACK chunk towards the client 102.
In the exemplary embodiment, the front-end server 106 receives the SCTP INIT-ACK chunk and inspects the contents of the INIT-ACK chunk to create a distribution key to manage the communications between the initiating client 102 and the selected back-end server 108. The exemplary embodiment continues with the front-end server 106 combining the client 102 SCTP port number with the Initiate-Tag and the back-end server 108 SCTP port number to create a distribution key for the SCTP association. Continuing with the exemplary embodiment, the front-end server 106 checks the verification tag translation table to confirm that the newly created distribution key is not already in use by another SCTP association managed by the front-end server 106. In the exemplary embodiment, if the distribution key is found in the verification tag translation table then the front-end server 106 generates a new Initiate-Tag and creates a non-clashing distribution key.
Next in the exemplary embodiment, the front-end server creates a new entry in the verification tag translation table for the client 102 and back-end server 108 generated Initiate-Tags and stores the values in the verification tag translation table. Continuing with the exemplary embodiment, the front-end server 106 updates the SCTP INIT-ACK chunk with the new Initiate-Tag and a recalculated checksum and, at step 408, forwards the updated SCTP INIT-ACK chunk to the client 102. It should be noted in the exemplary embodiment that if the front-end server 106 does not detect a clash of distribution keys then the front-end server 106 does not create an entry in the verification tag translation table for the SCTP association.
Continuing at step 410 of the exemplary environment, the client 102 sends a COOKIE-ECHO chunk towards the back-end server 108 and the intermediate front-end server 106 inspects the COOKIE-ECHO chunk to determine if the distribution key is a match with any of the distribution keys stored in the verification tag translation table. In the exemplary embodiment, if the distribution key matches an entry of the verification tag translation table then the front-end server 106 replaces the Verification-Tag in the COOKIE-ECHO chunk with the Initiate tag from the verification tag translation table, replaces the checksum with a checksum recalculated based on the replaced Verification-Tag and, at step 412, forwards the COOKIE-ECHO chunk to the back-end server 108. Next in the exemplary embodiment at 414, the back-end server 108 sends a COOKIE-ACK chunk towards the client 102 and at step 416 the front-end server 106 transparently forwards the COOKIE-ACK chunk towards the client 102.
Turning now to FIG. 5, illustrated is an exemplary embodiment 500. The exemplary embodiment 500 depicts the signaling flow for a back-end server 108 initiating an SCTP association with a client 102 through a front-end server 106. It should be noted in the exemplary embodiment that the front-end server 106 and one or more back-end servers 108 share a virtual internet protocol (IP) address and the back-end servers 108 operate independently of each other. It should be further noted in the exemplary embodiment that the operation of the front-end server 106 is transparent to both the client 102 and the back-end server 108 involved in the SCTP association.
First, in the exemplary embodiment, the back-end server 108 generates an Initiate-Tag and sends the Initiate-Tag, at step 502, in an SCTP INIT chunk towards the client 102 transparently through the front-end server 106. Next in the exemplary embodiment, the front-end server 106 receives the SCTP INIT chunk from the back-end server 108 and transparently inspects the contents of the INIT chunk to create a distribution key to manage the communications between the destination client 102 and the initiating back-end server 108. The exemplary embodiment continues with the front-end server 106 combining the client SCTP port number with the back-end server 108 generated Initiate-Tag and the back-end server 108 SCTP port number to create a distribution key for the SCTP association.
Continuing with the exemplary embodiment, the front-end server 106 checks the verification tag translation table to confirm that the newly created distribution key is not already in use by another SCTP association managed by the front-end server 106. In the exemplary embodiment, if the distribution key is found in the verification tag translation table then the front-end server 106 generates a new Initiate-Tag to replace the back-end server 108 generated Initiate-Tag and creates a non-clashing distribution key. Next in the exemplary embodiment, the front-end server creates a new entry in the verification tag translation table for the client 102 and back-end server 108 generated Initiate-Tag and SCTP port numbers and stores the values in the verification tag translation table.
Continuing at step 504 with the exemplary embodiment, the front-end server 106 forwards the SCTP INIT chunk to the client 102 and the client 102 processes the SCTP INIT chunk by generating an SCTP INIT-ACK chunk including a client generated Initiate-Tag and a cookie associated with the client and, at step 506, sends the INIT-ACK chunk towards the back-end server 108 through the front-end server 106.
Next in the exemplary embodiment, the front-end server 106 receives the SCTP INIT-ACK chunk from the client 102 and transparently inspects the contents of the SCTP packet common header to retrieve the distribution key used to distribute the SCTP INIT-ACK to the appropriate back-end server 108. Continuing with the exemplary embodiment, the front-end server 106 checks the verification tag translation table to determine if the distribution key is in the verification tag translation table. In the exemplary embodiment, if the distribution key is found in the verification tag translation table then the front-end server 106 replaces the Verification-Tag in the SCTP common header of the INIT-ACK message with the associated back-end server 108 Initiate-Tag from the verification tag translation table and updates the checksum before forwarding the SCTP INIT-ACK to the appropriate back-end server 108 at step 508. It should be noted in the exemplary embodiment that if the front-end server 106 does not detect a clash of distribution keys then the front-end server 106 simply forwards the SCTP INIT-ACK to the appropriate back-end server 108 based on the Verification-Tag retrieved from the SCTP common header and the back-end server establishes an SCTP association with the client.
Continuing at step 510 of the exemplary environment, the back-end server 108 sends a COOKIE-ECHO chunk towards the client 102 through the front-end server 106 and the front-end server 106 transparently forwards, at step 512, the COOKIE-ECHO to the client 102 and the client establishes an SCTP association with the back-end server 108. Next in the exemplary embodiment at 514, the client 102 sends a COOKIE-ACK chunk towards the back-end server 108 and at step 516 the front-end server 106 determines if a distribution key exists for this SCTP association and accordingly if an exchange of Verification-Tags is required. The exemplary embodiment continues with the front-end server 106 transparently, with regard to the client 102 and the back-end server 108, forwarding the COOKIE-ACK chunk towards the back-end server 108.
Turning now to FIG. 6, illustrated is an exemplary embodiment 600. The exemplary embodiment 600 depicts the signaling flow for a client 102 communicating through a front-end server 106 to a back-end server 108 using an established SCTP association. It should be noted in the exemplary embodiment that the front-end server 106 and one or more back-end servers 108 share a virtual internet protocol (IP) address and the back-end servers 108 operate independently of each other. It should be further noted in the exemplary embodiment that the operation of the front-end server 106 is transparent to both the client 102 and the back-end server 108 involved in the SCTP association.
Next in the exemplary embodiment, a client 102 sends, at step 602, an SCTP packet through the front end server 106 towards a back-end server 108. The front-end server 106 receives the SCTP packet from the client 102 and transparently inspects the contents of the SCTP packet to retrieve the distribution key used to distribute the SCTP packet to the appropriate back-end server 108. Continuing with the exemplary embodiment, the front-end server 106 checks the verification tag translation table to determine if the distribution key is in the verification tag translation table. In the exemplary embodiment, if the distribution key is found in the verification tag translation table then the front-end server 106 replaces the Verification-Tag in the SCTP packet common header with the associated back-end server 108 Initiate-Tag from the verification tag translation table and updates the checksum before forwarding the SCTP packet to the appropriate back-end server 108 at step 604. It should be noted in the exemplary embodiment that if the front-end server 106 does not detect a clash of distribution keys then the front-end server 106 forwards the SCTP packet to the appropriate back-end server 108 based on the Verification-Tag retrieved from the SCTP packet common header.
Continuing at FIG. 7, an exemplary method embodiment 700 for providing SCTP communication traffic distribution based on SCTP verification tags. Starting at step 702, the exemplary method embodiment 700 can receive a request to initiate an SCTP association between a client 102 and a back-end server 108. In the exemplary embodiment the initiation request can come from the client 102 or the back-end server. It should be noted in the exemplary embodiment that method 700 operates on a front-end server 106 that is transparent to both the client 102 and the back-end server 108. Further, it should be noted in the exemplary that there are a plurality of back-end servers 108 that operate independently of each other and are not aware of the existence of each other.
Continuing with the exemplary embodiment at step 704, the front-end server 106 can generate a distribution key for routing SCTP communication packets between the client 102 and the back-end server 108 based on SCTP parameters collected from the client and the back-end server 108. Continuing with the exemplary embodiment, the front-end server 106 can use the SCTP port number from the client 102, the SCTP port number from the back-end server 108 and the SCTP Verification-Tag from the back-end server to generate a distribution tag that for this SCTP association.
Further, at step 706 of the exemplary embodiment, the method 700 transparently routes SCTP packets between the client 102 and the back-end server 108. The exemplary method embodiment 700 retrieves the distribution key from the SCTP packet and extracts the SCTP Verification-Tag from the distribution key. The SCTP Verification-Tag represents the identity of the client 102 or back-end server 108 destination connected to the SCTP association for this SCTP communication. The exemplary method embodiment 700 then sends this SCTP packet towards the identified client 102 or back-end server 108.
FIG. 8 illustrates an example of a suitable computing system environment 800 in which the claimed subject matter can be implemented, although as made clear above, the computing system environment 800 is only one example of a suitable computing environment for an exemplary embodiment and is not intended to suggest any limitation as to the scope of use or functionality of the claimed subject matter. Further, the computing environment 800 is not intended to suggest any dependency or requirement relating to the claimed subject matter and any one or combination of components illustrated in the example computing environment 800.
Looking now to FIG. 8, an example of a device for implementing the previously described innovation includes a general purpose computing device in the form of a computer 810. Components of computer 810 can include, but are not limited to, a processing unit 820, a system memory 830, and a system bus 890 that couples various system components including the system memory to the processing unit 820. The system bus 890 can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
Computer 810 can include a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 810. By way of example, and not limitation, computer readable media can comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile as well as removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CDROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 810. Communication media can embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and can include any suitable information delivery media.
The system memory 830 can include computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) and/or random access memory (RAM). A basic input/output system (BIOS), containing the basic routines that help to transfer information between elements within computer 810, such as during start-up, can be stored in memory 830. Memory 830 can also contain data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 820. By way of non-limiting example, memory 830 can also include an operating system, application programs, other program modules, and program data.
The computer 810 can also include other removable/non-removable and volatile/nonvolatile computer storage media. For example, computer 810 can include a hard disk drive that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive that reads from or writes to a removable, nonvolatile magnetic disk, and/or an optical disk drive that reads from or writes to a removable, nonvolatile optical disk, such as a CD-ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM and the like. A hard disk drive can be connected to the system bus 890 through a non-removable memory interface such as an interface, and a magnetic disk drive or optical disk drive can be connected to the system bus 890 by a removable memory interface, such as an interface.
A user can enter commands and information into the computer 810 through input devices such as a keyboard or a pointing device such as a mouse, trackball, touch pad, and/or other pointing device. Other input devices can include a microphone, joystick, game pad, satellite dish, scanner, or similar devices. These and/or other input devices can be connected to the processing unit 820 through user input 840 and associated interface(s) that are coupled to the system bus 890, but can be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB).
A graphics subsystem can also be connected to the system bus 890. In addition, a monitor or other type of display device can be connected to the system bus 890 through an interface, such as output interface 850, which can in turn communicate with video memory. In addition to a monitor, computers can also include other peripheral output devices, such as speakers and/or printing devices, which can also be connected through output interface 850.
The computer 810 can operate in a networked or distributed environment using logical connections to one or more other remote computers, such as remote server 870, which can in turn have media capabilities different from device 810. The remote server 870 can be a personal computer, a server, a router, a network PC, a peer device or other common network node, and/or any other remote media consumption or transmission device, and can include any or all of the elements described above relative to the computer 810. The logical connections depicted in FIG. 8 include a network 880, such as a local area network (LAN) or a wide area network (WAN), but can also include other networks/buses.
When used in a LAN networking environment, the computer 810 is connected to the LAN 880 through a network interface or adapter. When used in a WAN networking environment, the computer 810 can include a communications component, such as a modem, or other means for establishing communications over a WAN, such as the Internet. A communications component, such as a modem, which can be internal or external, can be connected to the system bus 890 through the user input interface at input 840 and/or other appropriate mechanism.
In a networked environment, program modules depicted relative to the computer 810, or portions thereof, can be stored in a remote memory storage device. It should be noted that the network connections shown and described are exemplary and other means of establishing a communications link between the computers can be used.
Additionally, it should be noted that as used in this application, terms such as “component,” “display,” “interface,” and other similar terms are intended to refer to a computing device, either hardware, a combination of hardware and software, software, or software in execution as applied to a computing device implementing a virtual keyboard. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program and a computing device. As an example, both an application running on a computing device and the computing device can be components. One or more components can reside within a process and/or thread of execution and a component can be localized on one computing device and/or distributed between two or more computing devices, and/or communicatively connected modules. Further, it should be noted that as used in this application, terms such as “system user,” “user,” and similar terms are intended to refer to the person operating the computing device referenced above.
Further, the term to “infer” or “inference” refer generally to the process of reasoning about or inferring states of the system, environment, user, and/or intent from a set of observations captured from events and/or data. Captured events and data can include user data, device data, environment data, behavior data, application data, implicit and explicit data, etc. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The inference can be probabilistic in that the computation of a probability distribution over states of interest based on a consideration of data and events. Inference can also refer to techniques employed for composing higher-level events from a set of events and/or data. Such inference results in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources.
The above-described exemplary embodiments are intended to be illustrative in all respects, rather than restrictive, of the present innovation. Thus the present innovation is capable of many variations in detailed implementation that can be derived from the description contained herein by a person skilled in the art. All such variations and modifications are considered to be within the scope and spirit of the present innovation as defined by the following claims. No element, act, or instruction used in the description of the present application should be construed as critical or essential to the invention unless explicitly described as such. Also, as used herein, the article “a” is intended to include one or more items.

Claims (18)

The invention claimed is:
1. A method, stored in a memory and executing on a processor, for distributing Stream Control Transmission Protocol (SCTP) packets between a client and one of a plurality of back-end servers, the method comprising:
routing by a front-end server said SCTP packets, each of which includes a distribution key created based on SCTP parameters associated with said client and said one of said plurality of back-end servers wherein said plurality of back-end servers share a virtual internet protocol (VIP) address with said front-end server;
wherein said distribution key comprises:
an SCTP port number provided by said client;
an SCTP port number provided by said one of said plurality of back-end servers; and
an SCTP Initiate-Tag provided by said one of said plurality of back-end servers; and
wherein said front-end server detects a clash between a first distribution key created for a first back-end server SCTP association and a second distribution key, while creating said second distribution key, for a second back-end server SCTP association and generates a replacement SCTP Initiate-Tag for said second distribution key to preserve the uniqueness of said first distribution key and said second distribution key.
2. The method of claim 1, wherein said distribution key is generated by said front-end server.
3. The method of claim 1, wherein said front-end server manages said replacement SCTP Initiate-Tag by maintaining a verification tag translation table to hold said replacement SCTP Initiate-Tag and said second distribution key as a matched pair.
4. The method of claim 1, wherein said front-end server updates a SCTP common header associated with said second distribution key based on said replacement SCTP Initiate-Tag and recalculating a checksum associated with said SCTP common header.
5. The method of claim 1, wherein said front-end server operates transparently with regard to said plurality of back-end servers.
6. The method of claim 1, wherein said plurality of back-end servers, sharing said VIP address, operate independently of each other and are not aware of the existence of each other.
7. The method of claim 1, wherein said method does not require Stream Control Transmission Protocol (SCTP) modifications.
8. The method of claim 1, wherein said method does not require Internet Protocol (IP) header modifications.
9. The method of claim 1, wherein said Initiate-Tag is configured to be the address of said one of said plurality of back-end servers.
10. A front end server for distributing Stream Control Transmission Protocol (SCTP) packets between a client and one of a plurality of back-end servers, comprising a processor and a memory, said memory containing instructions executable by said processor whereby said processor is operative to:
route said SCTP packets, each of which includes a distribution key created based on SCTP parameters associated with said client and said one of said plurality of back-end servers wherein said plurality of back-end servers share a virtual internet protocol (VIP) address with said front-end server;
wherein said distribution key comprises:
an SCTP port number provided by said client; an SCTP port number provided by said one of said plurality of back-end servers; and an SCTP Initiate-Tag provided by said one of said plurality of back-end servers; and
wherein said front-end server detects a clash between a first distribution key created for a first back-end server SCTP association and a second distribution key, while creating said second distribution key, for a second back-end server SCTP association and generates a replacement SCTP Initiate-Tag for said second distribution key to preserve the uniqueness of said first distribution key and said second distribution key.
11. The front end server of claim 10, wherein said distribution key is generated by said front-end server.
12. The front end server of claim 10, wherein said front-end server manages said replacement SCTP Initiate-Tag by maintaining a verification tag translation table to hold said replacement SCTP Initiate-Tag and said second distribution key as a matched pair.
13. The front end server of claim 10, wherein said front-end server updates a SCTP common header associated with said second distribution key based on said replacement SCTP Initiate-Tag and recalculating a checksum associated with said SCTP common header.
14. The front end server of claim 10, wherein said front-end server operates transparently with regard to said plurality of back-end servers.
15. The front end server of claim 10, wherein said plurality of back-end servers, sharing said VIP address, operate independently of each other and are not aware of the existence of each other.
16. The front end server of claim 10, wherein no Stream Control Transmission Protocol (SCTP) modifications are required.
17. The front end server of claim 10, wherein no Internet Protocol (IP) header modifications are required.
18. The front end server of claim 10, wherein said Initiate-Tag is configured to be the address of said one of said plurality of back-end servers.
US13/044,128 2011-03-09 2011-03-09 Load balancing SCTP associations using VTAG mediation Expired - Fee Related US8737210B2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US13/044,128 US8737210B2 (en) 2011-03-09 2011-03-09 Load balancing SCTP associations using VTAG mediation
PCT/IB2012/051096 WO2012120473A1 (en) 2011-03-09 2012-03-08 Load balancing sctp associations using vtag mediation
CN201280022601.1A CN103814565A (en) 2011-03-09 2012-03-08 Load balancing SCTP associations using VTAG mediation
EP12711006.2A EP2684339A1 (en) 2011-03-09 2012-03-08 Load balancing sctp associations using vtag mediation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/044,128 US8737210B2 (en) 2011-03-09 2011-03-09 Load balancing SCTP associations using VTAG mediation

Publications (2)

Publication Number Publication Date
US20120230187A1 US20120230187A1 (en) 2012-09-13
US8737210B2 true US8737210B2 (en) 2014-05-27

Family

ID=45894607

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/044,128 Expired - Fee Related US8737210B2 (en) 2011-03-09 2011-03-09 Load balancing SCTP associations using VTAG mediation

Country Status (4)

Country Link
US (1) US8737210B2 (en)
EP (1) EP2684339A1 (en)
CN (1) CN103814565A (en)
WO (1) WO2012120473A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9332053B2 (en) * 2012-06-15 2016-05-03 Tekelec, Inc. Methods, systems, and computer readable media for load balancing stream control transmission protocol (SCTP) messages
WO2018192655A1 (en) 2017-04-20 2018-10-25 Nokia Solutions And Networks Oy Sctp offloading
US10999202B2 (en) 2018-11-30 2021-05-04 Oracle International Corporation Methods, systems, and computer readable media for distributing Sigtran connections among signal transfer point (STP) message processors
US11576072B2 (en) 2020-09-21 2023-02-07 Oracle International Corporation Methods, systems, and computer-readable media for distributing S1 connections to mobility management entities (MMEs) and N2 connections to access and mobility management functions (AMFs)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9225638B2 (en) 2013-05-09 2015-12-29 Vmware, Inc. Method and system for service switching using service tags
US11296930B2 (en) 2014-09-30 2022-04-05 Nicira, Inc. Tunnel-enabled elastic service model
US10135737B2 (en) 2014-09-30 2018-11-20 Nicira, Inc. Distributed load balancing systems
US9755898B2 (en) 2014-09-30 2017-09-05 Nicira, Inc. Elastically managing a service node group
US10594743B2 (en) 2015-04-03 2020-03-17 Nicira, Inc. Method, apparatus, and system for implementing a content switch
CN109429220B (en) * 2017-06-20 2022-03-22 中兴通讯股份有限公司 Multi-system co-coupling method and device
CN107243156B (en) * 2017-06-30 2020-12-08 珠海金山网络游戏科技有限公司 Large-scale distributed network game server system
US10797966B2 (en) 2017-10-29 2020-10-06 Nicira, Inc. Service operation chaining
US11012420B2 (en) 2017-11-15 2021-05-18 Nicira, Inc. Third-party service chaining using packet encapsulation in a flow-based forwarding element
US10797910B2 (en) 2018-01-26 2020-10-06 Nicira, Inc. Specifying and utilizing paths through a network
US10659252B2 (en) 2018-01-26 2020-05-19 Nicira, Inc Specifying and utilizing paths through a network
US10728174B2 (en) 2018-03-27 2020-07-28 Nicira, Inc. Incorporating layer 2 service between two interfaces of gateway device
US10805192B2 (en) 2018-03-27 2020-10-13 Nicira, Inc. Detecting failure of layer 2 service using broadcast messages
US10944673B2 (en) 2018-09-02 2021-03-09 Vmware, Inc. Redirection of data messages at logical network gateway
US11595250B2 (en) 2018-09-02 2023-02-28 Vmware, Inc. Service insertion at logical network gateway
US11086654B2 (en) 2019-02-22 2021-08-10 Vmware, Inc. Providing services by using multiple service planes
US11140218B2 (en) 2019-10-30 2021-10-05 Vmware, Inc. Distributed service chain across multiple clouds
US11283717B2 (en) 2019-10-30 2022-03-22 Vmware, Inc. Distributed fault tolerant service chain
US11223494B2 (en) 2020-01-13 2022-01-11 Vmware, Inc. Service insertion for multicast traffic at boundary
US11153406B2 (en) 2020-01-20 2021-10-19 Vmware, Inc. Method of network performance visualization of service function chains
US11659061B2 (en) 2020-01-20 2023-05-23 Vmware, Inc. Method of adjusting service function chains to improve network performance
US11438257B2 (en) 2020-04-06 2022-09-06 Vmware, Inc. Generating forward and reverse direction connection-tracking records for service paths at a network edge
US11611625B2 (en) 2020-12-15 2023-03-21 Vmware, Inc. Providing stateful services in a scalable manner for machines executing on host computers
US11734043B2 (en) 2020-12-15 2023-08-22 Vmware, Inc. Providing stateful services in a scalable manner for machines executing on host computers

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003041355A1 (en) 2001-11-05 2003-05-15 Nokia Corporation A method and a system for stateless load sharing for a server cluster in an ip-based telecommunications network
US6917626B1 (en) * 1999-11-30 2005-07-12 Cisco Technology, Inc. Apparatus and method for automatic cluster network device address assignment
US20070091902A1 (en) 2005-10-24 2007-04-26 Stewart Randall R Securely managing network element state information in transport-layer associations
US20080101357A1 (en) * 2006-10-31 2008-05-01 Paola Iovanna Method and apparatus for ip network interfacing
US20090265467A1 (en) * 2008-04-17 2009-10-22 Radware, Ltd. Method and System for Load Balancing over a Cluster of Authentication, Authorization and Accounting (AAA) Servers
WO2009129835A1 (en) 2008-04-22 2009-10-29 Telefonaktiebolaget L M Ericsson (Publ) Network node and method of routing messages in an ip-based signaling network

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100388721C (en) * 2003-03-18 2008-05-14 中兴通讯股份有限公司 Method for protecting coupling based on flow control transfer protocol
CN101515840B (en) * 2009-03-09 2011-08-03 北京交通大学 Route selecting method of multipath parallel transmission and delivery

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6917626B1 (en) * 1999-11-30 2005-07-12 Cisco Technology, Inc. Apparatus and method for automatic cluster network device address assignment
WO2003041355A1 (en) 2001-11-05 2003-05-15 Nokia Corporation A method and a system for stateless load sharing for a server cluster in an ip-based telecommunications network
US20040197079A1 (en) * 2001-11-05 2004-10-07 Nokia Corporation Method and a system for stateless load sharing for a server cluster in an IP-based telecommunications network
US20070091902A1 (en) 2005-10-24 2007-04-26 Stewart Randall R Securely managing network element state information in transport-layer associations
US20080101357A1 (en) * 2006-10-31 2008-05-01 Paola Iovanna Method and apparatus for ip network interfacing
US20090265467A1 (en) * 2008-04-17 2009-10-22 Radware, Ltd. Method and System for Load Balancing over a Cluster of Authentication, Authorization and Accounting (AAA) Servers
WO2009129835A1 (en) 2008-04-22 2009-10-29 Telefonaktiebolaget L M Ericsson (Publ) Network node and method of routing messages in an ip-based signaling network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
International Search Report for PCT/IB2012/051096 with mailing date of Jun. 12, 2012; 4 pages.

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9332053B2 (en) * 2012-06-15 2016-05-03 Tekelec, Inc. Methods, systems, and computer readable media for load balancing stream control transmission protocol (SCTP) messages
WO2018192655A1 (en) 2017-04-20 2018-10-25 Nokia Solutions And Networks Oy Sctp offloading
US10999202B2 (en) 2018-11-30 2021-05-04 Oracle International Corporation Methods, systems, and computer readable media for distributing Sigtran connections among signal transfer point (STP) message processors
US11576072B2 (en) 2020-09-21 2023-02-07 Oracle International Corporation Methods, systems, and computer-readable media for distributing S1 connections to mobility management entities (MMEs) and N2 connections to access and mobility management functions (AMFs)

Also Published As

Publication number Publication date
WO2012120473A1 (en) 2012-09-13
CN103814565A (en) 2014-05-21
WO2012120473A9 (en) 2014-03-13
EP2684339A1 (en) 2014-01-15
US20120230187A1 (en) 2012-09-13

Similar Documents

Publication Publication Date Title
US8737210B2 (en) Load balancing SCTP associations using VTAG mediation
US10027712B2 (en) System and method for distributed load balancing with distributed direct server return
CN109937401B (en) Live migration of load-balancing virtual machines via traffic bypass
US7774473B2 (en) System and method for sticky routing of requests within a server farm
US9338225B2 (en) Forwarding policies on a virtual service network
US8510447B2 (en) Direct addressability and direct server return
US9843630B2 (en) System and method for distributed load balancing with load balancer clients for hosts
US10375193B2 (en) Source IP address transparency systems and methods
CN112673596A (en) Service insertion at a logical gateway
US20130297798A1 (en) Two level packet distribution with stateless first level packet distribution to a group of servers and stateful second level packet distribution to a server within the group
CN110166570B (en) Service session management method and device, and electronic device
US20120036180A1 (en) Service virtualization over content-centric networks
US20110185082A1 (en) Systems and methods for network virtualization
US10826832B2 (en) Load balanced access to distributed scaling endpoints using global network addresses
US20090299937A1 (en) Method and system for detecting and managing peer-to-peer traffic over a data network
US20120233240A1 (en) Sctp association endpoint relocation in a load balancing system
US7136385B2 (en) Method and system for performing asymmetric address translation
US20170085523A1 (en) Methods for nat (network address translation) traversal and systems using the same
US9055117B1 (en) Distributed network address translation
US11652736B2 (en) Transmitting network traffic to a pool of redundant network appliances
US9509600B1 (en) Methods for providing per-connection routing in a virtual environment and devices thereof
US20120072612A1 (en) Method and an Arrangement of Identifying Traffic Flows in a Communication Network
US20230370421A1 (en) Scaling ip addresses in overlay networks
US10341292B2 (en) Increased port address space
US7844731B1 (en) Systems and methods for address spacing in a firewall cluster

Legal Events

Date Code Title Description
AS Assignment

Owner name: TELEFONAKTIEBOLAGET L M ERICSSON (PUBL), SWEDEN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TREMBLAY, RICHARD;CHATILA, ABDALLAH;REEL/FRAME:029810/0626

Effective date: 20110414

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551)

Year of fee payment: 4

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20220527