US20130208621A1 - Methods of redirecting network forwarding elements and related forwarding elements and controllers - Google Patents
Methods of redirecting network forwarding elements and related forwarding elements and controllers Download PDFInfo
- Publication number
- US20130208621A1 US20130208621A1 US13/369,993 US201213369993A US2013208621A1 US 20130208621 A1 US20130208621 A1 US 20130208621A1 US 201213369993 A US201213369993 A US 201213369993A US 2013208621 A1 US2013208621 A1 US 2013208621A1
- Authority
- US
- United States
- Prior art keywords
- network
- forwarding element
- network controller
- controller
- openflow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/64—Routing or path finding of packets in data switching networks using an overlay routing layer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/44—Distributed routing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/54—Store-and-forward switching systems
- H04L12/56—Packet switching systems
Definitions
- the present disclosure is directed to communications and, more particularly, to network communications and related methods, network forwarding elements, and network controllers.
- Ethernet forwarding elements e.g., switches and routers
- flow-tables typically built from TCAMs or Ternary Content Addressable Memories
- NAT network address translation
- QoS quality of service
- OpenFlow provides an open protocol to program flowtables in different forwarding elements (e.g., switches and routers).
- a network administrator for example, can partition traffic into production and research flows, and/or researchers can control their own flows by choosing the routes their packets follow and the processing they receive. In this way, researchers can try new routing protocols, security models, addressing schemes, and even alternatives to IP (Internet Protocol).
- IP Internet Protocol
- the production traffic may be isolated and processed conventionally.
- the datapath of an OpenFlow forwarding element may include a flow table, and an action associated with each flow entry included in the flow table.
- the set of actions supported by an OpenFlow forwarding element may be extensible. For high-performance and low-cost, the datapath may have a carefully prescribed degree of flexibility, which may mean forgoing the ability to specify arbitrary handling of each packet and seeking a more limited, but still useful, range of actions.
- An OpenFlow forwarding element may include a flow table having a plurality of flow entries (with an action associated with each flow entry) to tell the forwarding element how to process the respective flow, a secure channel that connects the switch to a remote OpenFlow controller (allowing commands and packets to be sent between the controller and the forwarding element using the OpenFlow Protocol (which provides an open and standard way for a controller to communicate with a forwarding element).
- a standard interface the OpenFlow Protocol
- researchers may not need to individually program OpenFlow forwarding elements.
- An OpenFlow forwarding element may include one or more flow tables and a group table (which may perform packet lookups and forwarding) and an OpenFlow channel to an external OpenFlow controller.
- the OpenFlow controller manages the forwarding element via the OpenFlow protocol. Using this protocol, the controller can add, update, and delete flow entries, both reactively (in response to packets received at the forwarding element) and proactively (e.g., to program flow tables of a new forwarding element).
- Each flow table in the forwarding element may include a set of flow entries.
- Each flow entry may include matched fields, counters, and a set of instructions to apply to matching packets.
- Matching at a forwarding element may start at a first flow table and may continue to additional flow tables of the forwarding element.
- Flow entries match data packets in priority order, with the first matching entry in each table being used. If a matching entry is found for a data packet in a flow table, the instructions associated with the specific flow entry are executed for the data packet. If no match is found for the data packet in a flow table, the outcome may depend on forwarding element configuration.
- the data packet may be forwarded to the controller over the OpenFlow channel, the data packet may be dropped, or attempts to match the data packet may continue to a next flow table of the forwarding element.
- Pipeline processing instructions allow data packets to be sent to subsequent tables for further processing and allow information (e.g., in the form of metadata) to be communicated between tables.
- Table pipeline processing may stop when the instruction set associated with a matching flow entry does not specify a next table. At this point, the data packet may usually be modified and forwarded.
- Flow entries may forward respective data packets to a port. This is usually a physical port, but it may also be a virtual port defined by the switch or a reserved virtual port defined by the OpenFlow switch specification. Reserved virtual ports may specify generic forwarding actions such as sending to the controller, flooding, or forwarding using non-OpenFlow methods, such as “normal” switch processing, while switch-defined virtual ports may specify link aggregation groups, tunnels or loopback interfaces.
- Flow entries may also point to a group, which specifies additional processing.
- Groups represent sets of actions for flooding, as well as more complex forwarding semantics (e.g., multipath, fast reroute, and link aggregation).
- groups also enable multiple flows to forward to a single identifier (e.g., IP forwarding to a common next hop). This abstraction may allow common output actions across flows to be changed efficiently.
- a group table may include group entries, with each group entry including a list of action buckets with specific semantics dependent on group type. The actions in one or more action buckets are applied to data packets sent to the group.
- OpenFlow forwarding elements e.g., switches and/or routers
- controllers e.g., controllers, and protocols
- OpenFlow Switch Specification Version 1.1.0 Implemented (Wire Protocol 0x02), Feb. 28, 2011, and in the reference by McKeown et al. entitled “OpenFlow: Enabling Innovation In Campus Networks,” Mar. 14, 2008.
- the disclosures of both of the above referenced documents are hereby incorporated herein in their entireties by reference.
- the OpenFlow channel is an interface that connects an OpenFlow forwarding element with a controller over an OpenFlow interface.
- the interface itself may be implementation specific, and it may be implemented using a TCP (Transmission Control Protocol) connection or a SCTP (Stream Control Transmission Protocol) connection.
- TLS Transport Layer Security
- TLS Transport Layer Security
- Control for a network of OpenFlow forwarding elements may be implemented using a cluster of OpenFlow controllers, and each forwarding element may use a known IP (Internet Protocol) address of one of the controllers to connect with the addressed controller according to a configuration protocol.
- IP Internet Protocol
- a non-standard configuration channel may be used to configure a connection between a forwarding element and a respective controller.
- a forwarding element may be programmed with a list of IP addresses for controllers to connect with, and the forwarding element may sequentially attempt to connect to a controller using each controller address in the list until a successful connection with a controller is made.
- a connection between an OpenFlow forwarding element and an OpenFlow controller may be set up responsive to the forwarding element initiating a connection socket with the controller and requesting the connection. The controller may then decide whether to allow the connection or not. If the controller accepts the connection, the connection may be completed and messages between the controller and the forwarding element may be transmitted over the resulting OpenFlow channel. If the controller does not accept the connection, the connection socket may be terminated.
- the OpenFlow controller may only be able to drop the connection by ignoring an ECHO-REQUEST communication from the forwarding element (i.e., by not transmitting an ECHO-REPLY in response to the ECHO-REQUEST) thereby allowing the connection to timeout. Dropping a connection in this manner, however, may be dependent on a timeout value configuration of the OpenFlow Request-Reply protocol.
- the controller decides to drop a connection with a forwarding element, the connection may not actually be dropped until after a next ECHO-REQUEST has been transmitted by the forwarding element and a timeout period has passed after the ECHO-REQUEST without transmitting an ECHO-REPLY.
- a complex configuration channel may be adopted, for example, to configure an OpenFlow forwarding element to connect with a controller from a list of controllers known to the forwarding element. Using such a known list of controllers, however, may make it difficult for a forwarding element to connect to an unknown controller (e.g., a controller added to the cluster after provisioning the list). In addition, it may be difficult to share the load of network forwarding elements in the forwarding plane among controllers in the control plane.
- one controller of a cluster of controllers may be designated as a master controller of the cluster, and only the master controller may be allowed to accept OpenFlow connections from network forwarding elements. If another controller is later designated among the controllers as the master controller or if the master controller fails in a conventional arrangement, the forwarding element may be delayed in connecting to a new master controller and/or the forwarding element may be unable to connect to the new master element.
- a method of connecting a network forwarding element may include transmitting a first connection request from the forwarding element to a first network controller. After transmitting the first connection request, a rejection message may be received from the first network controller at the network forwarding element, with the rejection message being responsive to the first connection request and with the rejection message including an address of a second network controller. Responsive to receiving the rejection message, a second connection request may be transmitted from the network forwarding element to the second network controller.
- a forwarding element may be efficiently connected to a desired controller even through the desired controller is not known to the forwarding element at the time of the connection request. Such redirection may thus facilitate connection of a new forwarding element to an appropriate controller in a network, connection of a forwarding element to a new master controller after loss of a previous master controller, and/or distribution of forwarding element loads across a plurality of peer controllers.
- the rejection message may include the address of the second network controller and an address of a third network controller.
- the first and second network controllers may be respective first and second OpenFlow network controllers
- the network forwarding element may be an OpenFlow network forwarding element
- the rejection message may be transmitted over an OpenFlow channel.
- the network forwarding element may include a flow table having a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream.
- an acceptance message may be received from the second network controller at the network forwarding element after transmitting the second connection request, and an instruction to change at least one of the plurality of flow entries may be received from the second network controller at the network forwarding element after receiving the acceptance message.
- the at least one of the plurality of flow entries may be changed responsive to the instruction received from the second network controller. After changing the at least one of the plurality of flow entries, a data packet of a data stream may be processed through the network forwarding element in accordance with the at least one of the plurality of flow entries including the change.
- the rejection message may include the address of the second network controller and an address of a third network controller, and a third connection request may be transmitted from the network forwarding element to the third network controller responsive to transmitting the second connection request without establishing a connection with the second network controller.
- the network forwarding element may include a flow table having a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream.
- an acceptance message may be received from the third network controller at the network forwarding element.
- an instruction to change at least one of the plurality of flow entries may be received from the third network controller at the network forwarding element.
- the at least one of the plurality of flow entries may be changed responsive to the instruction received from the third network controller.
- a method of redirecting a network forwarding element may include receiving a connection request at a first network controller from the network forwarding element. Responsive to receiving the connection request, the controller may determine whether to accept control of the forwarding element at the first network controller. Responsive to determining to not accept control of the forwarding element at the first network controller, a rejection message may be transmitted from the first network controller to the network forwarding element, with the rejection message including an address of a second network controller.
- the rejection message may include the address of the second network controller and an address of a third network controller.
- the first and second network controllers may be respective first and second OpenFlow network controllers
- the network forwarding element may be an OpenFlow network forwarding element
- the rejection message may be transmitted over an OpenFlow channel.
- the network forwarding element may include a flow table having a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream. Responsive to determining to accept control of the forwarding element at the first network controller, an acceptance message may be transmitted from the first network controller to the network forwarding element. After transmitting the acceptance message, an instruction to change at least one of the plurality of flow entries may be transmitted from the first network controller to the network forwarding element.
- a network forwarding element may include a channel interface configured to provide communications between the network forwarding element in a forwarding plane and a control plane including a plurality of controllers, and a processor coupled to the channel interface.
- the processor may be configured to transmit a first connection request through the channel interface to a first network controller of the control plane, to receive a rejection message from the first network controller through the channel interface after transmitting the first connection request.
- the rejection message may be responsive to the first connection request and the rejection message may include an address of a second network controller of the control plane.
- the processor may be configured to transmit a second connection request from the network forwarding element to the second network controller responsive to receiving the rejection message.
- the rejection message may include the address of the second network controller and an address of a third network controller.
- the first and second network controllers may be respective first and second OpenFlow network controllers
- the network forwarding element may be an OpenFlow network forwarding element
- the channel interface may be an OpenFlow secure channel interface
- the rejection message may be transmitted over an OpenFlow channel.
- the network forwarding element may further include a plurality of input ports configured to receive data packets of data streams from other forwarding elements, and a plurality of output ports configured to transmit the data packets of the data streams to other forwarding elements, with the processor being coupled between the input and output ports.
- a flow table may be coupled to the processor, with the flow table including a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream between respective ones of the input and output ports, between a respective one of the input ports and the control plane, and/or between the control plane and a respective one of the output ports.
- the processor may be further configured to receive an acceptance message from the second network controller through the channel interface after transmitting the second connection request, to receive an instruction to change at least one of the plurality of flow entries from the second network controller through the channel interface after receiving the acceptance message, and to change the at least one of the plurality of flow entries of the flow table responsive to the instruction received from the second network controller.
- the processor may be further configured to process a data packet of a data stream between respective ones of the input and output ports, between respective ones of the input ports and the control plane, and/or between the control plane and respective ones of the output ports in accordance with the at least one of the plurality of flow entries including the change.
- the rejection message may include the address of the second network controller and an address of a third network controller, with the processor being further configured to transmit a third connection request through the channel interface to the third network controller responsive to transmitting the second connection request without establishing a connection with the second network controller.
- the forwarding element may further include a plurality of input ports configured to receive data packets of data streams from other forwarding elements and a plurality of output ports configured to transmit the data packets of the data streams to other forwarding elements, with the processor being coupled between the input and output ports.
- a flow table may be coupled to the processor, with the flow table including a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream between respective ones of the input and output ports, between a respective one of the input ports and the control plane, and/or between the control plane and a respective one of the output ports.
- the processor may be further configured to receive an acceptance message from the third network controller through the channel interface after transmitting the third connection request, to receive an instruction to change at least one of the plurality of flow entries from the third network controller through the channel interface after receiving the acceptance message, and to change the at least one of the plurality of flow entries responsive to the instruction received from the third network controller.
- a network controller may include a channel interface configured to provide communications between the network controller in a control plane and a forwarding plane including a plurality of network forwarding element, and a processor coupled to the channel interface.
- the processor may be configured to receive a connection request through the channel interface from the network forwarding element, to determine whether to accept control of the forwarding element at the first network controller responsive to receiving the connection request, and to transmit a rejection message through the channel interface to the network forwarding element responsive to determining to not accept control of the forwarding element, with the rejection message including an address of a second network controller.
- the rejection message may include the address of the second network controller and an address of a third network controller.
- the first and second network controllers may be respective first and second OpenFlow network controllers
- the network forwarding element may be an OpenFlow network forwarding element
- the channel interface may be an OpenFlow secure channel interface
- the rejection message may be transmitted over an OpenFlow channel.
- the network forwarding element may include a flow table having a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream.
- the processor may be further configured to transmit an acceptance message through the channel interface to the network forwarding element responsive to determining to accept control of the forwarding element, and to transmit an instruction to change at least one of the plurality of flow entries through the channel interface to the network forwarding element after transmitting the acceptance message.
- FIG. 1 is a block diagram illustrating a network including controllers and forwarding elements according to some embodiments
- FIG. 2 is a block diagram illustrating a forwarding element of FIG. 1 according to some embodiments
- FIG. 3 is a block diagram illustrating a controller of FIG. 1 according to some embodiments.
- FIG. 4 is a flow chart illustrating operations of a forwarding element according to some embodiments.
- FIG. 5 is flow chart illustrating operations of a controller according to some embodiment.
- FIG. 6 is a flow diagram illustrating communications between a forwarding element and controllers according to some embodiments.
- the OpenFlow protocol may be extended to provide redirection of connection between an OpenFlow forwarding element in a network forwarding plane from one OpenFlow controller to another OpenFlow controller in a network control plane. Communications may be provided between the controllers so that each controller may be aware of the existence and/or status of the other controllers in the control plane. Accordingly, any controller in the control plane that receives a connection request from a forwarding element in the forwarding plane may redirect the forwarding element to another controller that may be more appropriate for connection with the forwarding element.
- FIG. 1 is a block diagram illustrating a network including a plurality of OpenFlow controllers 101 - 1 to 101 - n (controllers) in a control plane and a plurality of OpenFlow forwarding elements 103 - 1 to 103 - m (forwarding elements) in a forwarding plane, with communications between OpenFlow Controllers 101 - 1 to 101 - n and OpenFlow forwarding elements 103 - 1 to 103 - m being provided though an OpenFlow channel (a channel) therebetween in accordance with an OpenFlow protocol(s).
- OpenFlow controllers 101 - 1 to 101 - n controllers
- OpenFlow forwarding elements 103 - 1 to 103 - m forwarding elements
- the OpenFlow channel may be implemented, for example, using a Transmission Control Protocol (TCP) connection and/or a Stream Control Transmission Protocol (SCTP) connection, and Transport Layer Security (TLS) may be used to communicate encrypted messages between respective controllers and forwarding elements.
- TCP Transmission Control Protocol
- SCTP Stream Control Transmission Protocol
- TLS Transport Layer Security
- communication paths 111 may be provided between OpenFlow forwarding elements 103 - 1 to 103 - m in the forwarding plane to support transmission of data packets between forwarding elements.
- communication paths 115 (separate from the OpenFlow channel) may be provided between controllers 101 - 1 to 101 - n in the control plane to support coordination between controllers.
- FIG. 2 is a block diagram illustrating elements of an OpenFlow forwarding element 103 of FIG. 1 .
- each forwarding element 103 may include a processor 207 coupled between one or more input ports 205 and one or more output ports 209 .
- One or more flow tables 201 may be coupled with processor 207 , each flow table may include a plurality of flow entries, and each flow entry may define instructions used to process/forward data packets matching the respective flow entry. Stated in other words, each flow entry may define processing for data packets of a respective data stream(s).
- Data packets from other forwarding elements may be received at respective input ports 205 , processed by processor 207 in accordance with respective flow entries from flow table(s) 201 , and transmitted from respective output ports 209 to other forwarding elements.
- Each forwarding element 103 may also include memory 211 , group tables 215 , and OpenFlow Secure Channel Interface 203 (a channel interface). More particularly, OpenFlow secure channel interface 203 may provide communications between processor 207 and a respective OpenFlow controller 101 through the OpenFlow Channel of FIG. 1 . OpenFlow secure channel interface 203 , for example, may use TLS to encrypt/send messages over the OpenFlow Channel to a respective controller and/or to receive/decrypt messages transmitted by the respective controller over the OpenFlow Channel.
- processor 207 may transmit the data packet through OpenFlow Secure Channel Interface 203 over the OpenFlow Channel to a respective controller.
- processor 207 may receive messages from the respective controller over the OpenFlow Channel through OpenFlow Channel interface 203 to change (e.g., add, update, and/or delete) a flow entry(ies) of flow table(s) 201 .
- FIG. 3 is a block diagram illustrating elements of an OpenFlow controller 101 of FIG. 1 .
- each controller 101 may include a processor 307 coupled between OpenFlow Secure Channel Interface 303 (channel interface) and controller interface 309 .
- OpenFlow Secure Channel Interface 303 may provide communications between processor 307 and a respective OpenFlow forwarding element 103 through the OpenFlow Channel of FIG. 1 .
- OpenFlow secure channel interface 303 may use TLS to encrypt/send messages over the OpenFlow Channel to a respective forwarding element(s) and/or to receive/decrypt messages transmitted by the respective forwarding element(s) over the OpenFlow Channel.
- OpenFlow controller 101 may also include memory 311 coupled to processor 307 .
- OpenFlow forwarding element 103 (a forwarding element) and OpenFlow controller 101 (a controller) of FIGS. 1 , 2 , and 3 will be discussed in greater detail below with respect to the flow charts of FIGS. 4 and 5 . Because operations of forwarding element 103 and controller 101 of embodiments discussed herein are interrelated, operations of FIGS. 4 and 5 will be discussed together. Moreover, each forwarding element of FIG. 1 may have a structure as discussed above with respect FIG. 2 , and each controller of FIG. 1 may have a structure as discussed above with respect to FIG. 3 .
- each controller may maintain in memory 311 thereof a listing of addresses for all of the other controllers of the control plane, and this listing of addresses and/or other network information may be used by each controller to identify one or more other controllers to which forwarding elements may be redirected.
- forwarding element 103 memory 211 may be programmed with an address of a controller or a list of addresses of controllers with which it should initiate connection when coupled to the network.
- the controller address or addresses may be programmed during manufacturing and/or later using a default controller address or addresses. More particularly, the controller address(es) may be an Internet Protocol (IP) address(es).
- IP Internet Protocol
- processor 207 may detect that OpenFlow secure channel interface 203 (channel interface) is coupled to the OpenFlow channel.
- Processor 207 detecting communication with the control plane may serve as a triggering event for processor 207 to initiate connection to a controller of the control plane at block 401 of FIG. 4 .
- initial connection to the OpenFlow channel is discussed as an example of a triggering event, other events may trigger connection/reconnection at block 401 .
- loss of an existing connection with a controller may trigger connection/reconnection at block 401 .
- processor 207 may identify a controller address (e.g., an IP address of a controller) from memory 211 for network connection at block 403 of FIG. 4 . As discussed above, one controller address or a list of controller addresses may be saved in memory 211 . The only controller address or the first controller address of a list may be selected by processor 207 for network connection.
- a controller address e.g., an IP address of a controller
- processor 207 may transmit a connection request using the controller address selected from memory 211 .
- the controller address may identify controller 101 - 1 of FIG. 1 .
- processor 207 may initiate a socket connection between channel interface 203 of forwarding element 103 - 1 and channel interface 303 of controller 101 - 1 over the OpenFlow channel using the address of controller 101 - 1 obtained from memory 211 , and the connection request may be transmitted to controller 101 - 1 using the socket connection.
- processor 207 may attempt to establish a socket connection sequentially using each controller address of the list until a socket connection is established, and the communication request may be transmitted to the controller using the socket connection at block 405 .
- processor 307 may decide at block 503 of FIG. 5 whether to accept or reject the connection request from forwarding element 103 - 1 . If processor 307 accepts the connection request at block 503 , processor 307 may transmit an acceptance message at block 507 through channel interface 303 over the OpenFlow channel to forwarding element 103 - 1 . Responsive to receiving the acceptance message over the OpenFlow channel and through channel interface 203 at block 407 , processor 207 of forwarding element 103 - 1 may proceed with operations of blocks 421 to 427 of FIG. 4 , and after transmitting the acceptance message, processor 307 of controller 101 - 1 may proceed with operations 509 to 515 of FIG. 5 .
- processor 307 of controller 101 - 1 may determine if flow entries of flow table(s) 201 of forwarding element 103 - 1 should be changed (e.g., added, updated, and/or deleted). Processor 307 of controller 101 - 1 may decide to add one or more new flow entries, update one or more existing flow entries, and/or or delete one or more existing flow entries of flow table(s) 201 of forwarding element 103 - 1 . Such changes may be made proactively (e.g., to initially populate one or more flow tables, to balance loads across multiple forwarding elements, etc.) or reactively (e.g., to accommodate new data streams responsive to unmatched data packets at forwarding element 103 - 1 ).
- processor 307 may transmit a change instruction(s) through channel interface 303 over OpenFlow channel to forwarding element 103 - 1 at block 511 . Moreover, operations of blocks 509 and 511 may be repeated as long as the connection between controller 101 - 1 and forwarding element 103 - 1 is maintained at block 515 .
- processor 207 of forwarding element 103 - 1 may change (e.g., add, update, and/or delete) at least one flow entry of its flow table(s) responsive to the change instructions at block 423 , and processor 207 may process data packets through forwarding element 103 - 1 in accordance with the flow tables (including any changes implemented at block 423 ) at block 425 .
- operations of blocks 421 , 423 , and 425 may be repeated as long as the connection between controller 101 - 1 and forwarding element 103 - 1 is maintained at block 427 .
- processor 307 may forward the non-matching data packet(s) through channel interface 203 and over OpenFlow channel to controller 101 - 1 .
- processor 307 of controller 101 - 1 may decide to add or update a flow entry of flow table(s) 201 of forwarding element 103 - 1 to handle a new data stream including the non-matching data packet.
- Processor 307 of controller 101 - 1 may transmit the instruction at block 511 , processor 207 of forwarding element 103 - 1 may receive the instruction at block 421 , and processor 207 of forwarding element 103 - 1 may add/update a flow entry of flow table(s) 201 in accordance with the instruction. Processor 207 of forwarding element 103 - 1 may then process subsequent data packets of the new data stream in accordance with the added/updated flow entry.
- forwarding element 103 - 1 and controller 101 - 1 may proceed with operations 421 , 423 , 425 , and 427 of FIG. 4 and with operations 509 , 511 , and 515 of FIG. 5 .
- processor 307 of controller 101 - 1 may reject the connection request at block 503 and transmit a rejection message through channel interface 303 over the OpenFlow channel to forwarding element 103 - 1 at block 505 of FIG. 5 .
- the rejection message may include an address (e.g., an IP address) of a secondary controller (e.g., controller 101 - 2 ) or a list of addresses (e.g., IP addresses) of respective secondary controllers (e.g., controllers 101 - 2 , 101 - 3 , 101 - 4 , etc.).
- the rejection message may include an address/addresses for one or more secondary controllers that may be more suitable to control forwarding element 103 - 1 .
- the socket connection between forwarding element 103 - 1 and controller 101 - 1 may be dropped.
- processor 207 of forwarding element 103 - 1 may transmit a second connection request (through channel interface 203 over the OpenFlow channel) to a secondary controller (e.g., controller 101 - 2 ) using the first address from the list of addresses of secondary controllers if a list is provided (or using the only address if only one address is provided) at block 411 of FIG. 4 . More particularly, processor 207 may establish a socket connection between channel interface 203 of forwarding element 103 - 1 and channel interface 303 of secondary controller 101 - 2 over the OpenFlow channel using the address of controller 101 - 2 , and the second connection request may be transmitted over the socket connection.
- a secondary controller e.g., controller 101 - 2
- processor 207 may establish a socket connection between channel interface 203 of forwarding element 103 - 1 and channel interface 303 of secondary controller 101 - 2 over the OpenFlow channel using the address of controller 101 - 2 , and the second connection request may be transmitted over the socket connection.
- the secondary controller (e.g., controller 101 - 2 ) may then proceed with operations of FIG. 5 as discussed above with respect to primary controller 101 - 1 .
- processor 307 of secondary controller 101 - 2 may either accept or reject the connection request at block 503 , and either transmit an acceptance message at block 507 or a rejection message at block 505 .
- processor 307 of controller 101 - 2 transmits an acceptance message at block 507 and the acceptance message is received by processor 207 of forwarding element 103 - 1 at block 415 , processor 207 of forwarding element 103 - 1 may proceed with operations of blocks 421 , 423 , 425 , and 427 and processor 307 of controller 101 - 2 may proceed with operations of blocks 509 , 511 , and 515 to provide a connection between forwarding element 103 - 1 and controller 101 - 1 .
- the rejection message may include a secondary controller address or a list of secondary controller addresses to be used by processor 207 of forwarding element 103 - 1 for subsequent connection requests.
- a rejection message from a secondary controller may omit addresses of other secondary controllers so that the forwarding element 103 - 1 continues using secondary controller addresses provided in a list from the first rejection message from primary controller 101 - 1 .
- the socket connection between forwarding element 103 - 1 and controller 101 - 2 may be dropped.
- processor 207 of forwarding element 103 - 1 may determine if all addresses of secondary controllers have been exhausted at block 417 . Processor 207 of forwarding element 103 - 1 may thus repeat operations of blocks 411 , 415 , and 417 until either a connection with a secondary controller is established or connection attempts to connect to all identified secondary controllers fail.
- a first rejection message received at block 407 from primary controller 101 - 1 may include a list of addresses of secondary controllers, and processor 207 of forwarding element 103 - 1 may proceed through operations of blocks 411 , 415 , and 417 sequentially using addresses from the list provided with the first rejection message until either a connection request is accepted at block 415 or all of the addresses of the list have been exhausted at block 417 .
- each rejection message received at block 415 may include an address(es) of a secondary controller(s) that is used for a subsequent connection request at block 411 .
- FIG. 6 is a message flow diagram illustrating an example of communications between forwarding element 101 - 1 and controllers 101 - 1 , 101 - 2 , and 101 - 3 over the OpenFlow channel as shown in FIG. 1 according to operations of FIGS. 4 and 5 .
- Forwarding element 103 - 1 may have the structure illustrated in FIG. 2 and may operate in accordance with operations of the flow chart of FIG. 4 .
- controllers 101 - 1 , 101 - 2 , and 101 - 3 may separately have the structure of FIG. 3 and may operate independently in accordance with operations of the flow chart of FIG. 5 .
- operations from the flow charts of FIGS. 4 and 5 are provided in parenthesis in FIG. 6 to illustrate a correspondence of messages of FIG. 6 and operations of FIGS. 4 and 5 .
- forwarding element 103 - 1 may transmit a first connection request to controller 101 - 1 (acting as a primary controller) and the connection request may be received at controller 101 - 1 as discussed above with respect to operations 405 and 501 of FIGS. 4 and 5 .
- Forwarding element 103 - 1 may transmit the first connection request responsive to detecting initial connection to the OpenFlow channel or responsive to loss of a previous connection with another controller.
- controller 101 - 1 may transmit a rejection message to forwarding element 103 - 1 and the rejection message may be received at forwarding element 103 - 1 as discussed above with respect to operations 505 and 407 of FIGS. 4 and 5 .
- the rejection message may include addresses of first and second secondary controllers 101 - 2 and 101 - 3 to be used by forwarding element 103 - 1 for subsequent connection requests.
- forwarding element 103 - 1 may transmit a second connection request to first secondary controller 101 - 2 using the first address in the list provided with the rejection message as discussed above with respect to operation 411 of FIG. 4 .
- first secondary controller 101 - 2 may be out of service so that the second connection request is not received by the second controller and/or no acceptance or rejection message is transmitted by first secondary controller 101 - 2 . Accordingly, no acceptance or rejection message is received by forwarding element 103 - 1 at block 415 . In this case, failure to receive a response to the second connection request within a designated timeout period may be interpreted as a rejection at block 415 of FIG. 4 .
- forwarding element 103 - 1 may transmit a third connection request to second secondary controller 101 - 3 using the second address in the list provided with the rejection message from controller 101 - 1 as discussed above with respect to operation 411 of FIG. 4 , and the third connection request may be received at the second secondary controller 101 - 3 as discussed above with respect to operation 501 of FIG. 5 .
- controller 101 - 3 may transmit an acceptance message to forwarding element 103 - 1 and the acceptance message may be received at forwarding element 103 - 1 as discussed above with respect to operations 507 and 415 of FIGS. 4 and 5 .
- OpenFlow communications may then be provide over the OpenFlow channel between forwarding element 103 - 1 and controller 101 - 3 as discussed above with respect to operations 509 , 511 , and 515 of FIG. 5 and operations 421 , 423 , 425 , and 427 of FIG. 4 .
- the rejection message of operation 505 of FIGS. 5 and 6 may be provided as a new message type, such as an OFPT_CONNECTION_REJECT message having the following format:
- the header may be an OpenFlow header including the version, type, length, and xid (a unique identification for the message), and a transaction identification. The transaction identification may be required to match a reply from the forwarding element with a request.
- a transaction identification may not be required because a forwarding element reply may not be received for a rejection message.
- the “num_of controllers” field may indicate a number of controller IP addresses included in the rejection message, and the “ofpt_controller_ip_list” provides the list of secondary controller IP addresses.
- An initial connection request (e.g., an OFPT_CONNECTION_REJECT message) may be transmitted from forwarding element 103 - 1 at block 405 of FIG. 4 and received by controller 101 - 1 at block 501 of FIG. 5 , and controller 101 - 1 may then decide whether to accept the connection request as discussed above with respect to block 503 of FIG. 5 .
- controller 101 - 1 may respond with an OFPT_FEATURES_REQUEST message as an acceptance message, forwarding element 103 - 1 may respond with an OFPT_FEATURES_REPLY message, and then a finite state machine (FSM) of controller 101 - 1 may progress normally to complete the connection between forwarding element 103 - 1 and controller 101 - 1 .
- FSM finite state machine
- controller 101 - 1 may transmit a rejection message as discussed above with respect to operation 505 of FIG. 5 . More particularly, the rejection message may be provided as an OFPT_CONNECTION_REJECT message discussed above, with the “ofpt_controller_ip_list” field including a list of one or more IP addresses of secondary controllers (e.g., IP addresses for controllers 101 - 2 , 101 - 3 , 101 - 4 , etc.), with the “num_of_controllers” field set to indicate the number of IP addresses included in the list, and with the message type set to “OFPT_CONNECTION_REJECT.” Controller 101 - 1 may then send the rejection message to forwarding element 101 - 1 as discussed above with respect to block 505 of FIG.
- the rejection message may be provided as an OFPT_CONNECTION_REJECT message discussed above, with the “ofpt_controller_ip_list” field including a list of one or more IP addresses of secondary controllers (e.g., IP addresses
- the forwarding element 103 - 1 may also close the socket connection, and then use the controller IP addresses from the “ofpt_controller_ip_list” field for subsequent connection attempts in accordance with operations 411 , 415 , and 417 of FIG. 4 .
- Forwarding element 101 - 1 may also save the controller IP addresses from the “ofpt_controller_ip_list” field for use in the event that a subsequent connection with a secondary controller is lost.
- a cluster of controllers 101 - 1 to 101 - n may be provided in the control plane, and one of the controllers (e.g., controller 101 - 1 ) may be elected as a master of the cluster.
- the remaining controllers e.g., controllers 101 - 2 to 101 - n
- the secondary controllers 101 - 2 to 101 - n may need to elect a new master controller, and all forwarding elements 103 - 1 to 103 - n may need to reconnect with the new master controller.
- Remaining controllers 101 - 2 to 101 - n may elect a new master (e.g., controller 101 - 2 ) based on information obtained/provided over communication paths 115 , but forwarding elements 103 - 1 to 103 - m may be unaware of this election. Accordingly, one or more of forwarding elements 103 - 1 to 103 - m may attempt reconnection with the wrong controller.
- original master controller 101 - 1 may fail, and the remaining controllers 101 - 2 to 101 - n of the control plane may elect controller 101 - 2 as the new master controller.
- each of forwarding elements 103 - 1 to 103 - m may attempt to connect with one of the other controllers 101 - 2 to 101 - m without knowing that controller 101 - 2 has been elected as the new master controller. Accordingly, some of forwarding elements 103 - 1 to 103 - m may connect with one of non-master controllers 101 - 3 to 101 - n. In this situation, the non-master controllers 101 - 3 to 101 - n may use rejection messages (also referred to as redirection messages) to redirect forwarding elements 103 - 1 to 103 - m to the new master controller 101 - 2 .
- rejection messages also referred to as redirection messages
- all forwarding elements 103 - 1 to 103 - n may be initially connected to original master controller 101 - 1 when master controller fails. Upon timeouts after respective failure to receive respective ECHO_REPLY messages from original master controller 101 - 1 , each forwarding element 103 - 1 to 103 - n may attempt to connect with another controller in accordance with operations discussed above with respect to FIG. 4 .
- Forwarding element 103 - 1 may initiate connection with controller 101 - 3 .
- forwarding element 103 - 1 and controller 101 - 3 may follow operations 401 , 403 , 405 , and 407 of FIG. 4 and operations 501 , 503 , and 505 of FIG. 5 to redirect forwarding element 103 - 1 to new master controller 101 - 2 .
- controller 101 - 3 may transmit a rejection message at block 505 of FIG. 5 including an IP address of the new master controller 101 - 1 .
- forwarding element 103 - 1 may proceed with operation of 411 , 415 , and 417 of FIG. 4 to connect with the new master controller 101 - 2 .
- any of controllers 101 - 3 to 101 - n may use a rejection message including the IP address of the new master controller 101 - 2 to efficiently redirect forwarding elements to the new master controller.
- no one controller may be designated as a mater controller, and instead, all controllers 101 - 1 to 101 - n in the control plane may be peer controllers, with each forwarding element being connected to one of the peer controllers. Accordingly, peer controllers 101 - 1 to 101 - n may share the load of forwarding elements 103 - 1 to 103 - m. Moreover, controllers 101 - 1 to 101 - n may share data/state information over communication paths 115 .
- the controller may determine whether to accept or reject the connection request based on its load and/or based on loads of other controllers. If the controller determines that the connection with the forwarding element is best handled by another controller (e.g., because its load is relatively high and/or because a load of another controller is relatively low), the controller may transmit a rejection message including an IP address of another controller (e.g., another controller that is currently experiencing a lower load). Rejection messages may thus be used to share forwarding element loads across controllers of the control plane.
- the controller may determine whether to accept or reject the connection request based on its load and/or based on loads of other controllers. If the controller determines that the connection with the forwarding element is best handled by another controller (e.g., because its load is relatively high and/or because a load of another controller is relatively low), the controller may transmit a rejection message including an IP address of another controller (e.g., another controller that is currently experiencing a lower load). Rejection messages may thus be used to share forwarding element loads across controllers of the control
- a plurality of controllers may be deployed in the control plane, and rejection messages may be used to control/redirect connections between forwarding elements of the forwarding plane and controllers of the control plane.
- new forwarding elements and/or controllers may be deployed in the forwarding and control planes while maintaining network operations without requiring an elaborate configuration protocol.
- rejection messages may be used to manage maintenance of the network.
- the terms “comprise”, “comprising”, “comprises”, “include”, “including”, “includes”, “have”, “has”, “having”, or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof.
- the common abbreviation “e.g.”, which derives from the Latin phrase “exempli gratia,” may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item.
- the common abbreviation “i.e.”, which derives from the Latin phrase “id est,” may be used to specify a particular item from a more general recitation.
- Example embodiments are described herein with reference to block diagrams and/or flowchart illustrations of computer-implemented methods, apparatus (systems and/or devices) and/or computer program products. It is understood that a block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits.
- These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and/or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and/or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions/acts specified in the block diagrams and/or flowchart block or blocks, and thereby create means (functionality) and/or structure for implementing the functions/acts specified in the block diagrams and/or flowchart block(s).
- These computer program instructions may also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions/acts specified in the block diagrams and/or flowchart block or blocks.
- a tangible, non-transitory computer-readable medium may include an electronic, magnetic, optical, electromagnetic, or semiconductor data storage system, apparatus, or device. More specific examples of the computer-readable medium would include the following: a portable computer diskette, a random access memory (RAM) circuit, a read-only memory (ROM) circuit, an erasable programmable read-only memory (EPROM or Flash memory) circuit, a portable compact disc read-only memory (CD-ROM), and a portable digital video disc read-only memory (DVD/BlueRay).
- RAM random access memory
- ROM read-only memory
- EPROM or Flash memory erasable programmable read-only memory
- CD-ROM compact disc read-only memory
- DVD/BlueRay portable digital video disc read-only memory
- the computer program instructions may also be loaded onto a computer and/or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer and/or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the block diagrams and/or flowchart block or blocks.
- embodiments of the present invention may be embodied in hardware and/or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as “circuitry,” “a module” or variants thereof.
Abstract
Description
- The present disclosure is directed to communications and, more particularly, to network communications and related methods, network forwarding elements, and network controllers.
- Most modern Ethernet forwarding elements (e.g., switches and routers) include flow-tables (typically built from TCAMs or Ternary Content Addressable Memories) that run at line-rate to implement firewalls, NAT (network address translation), and QoS (quality of service), and to collect statistics. While flow-tables of different vendors may be different, OpenFlow exploits a common set of functions that run in many switches and routers.
- OpenFlow provides an open protocol to program flowtables in different forwarding elements (e.g., switches and routers). A network administrator, for example, can partition traffic into production and research flows, and/or researchers can control their own flows by choosing the routes their packets follow and the processing they receive. In this way, researchers can try new routing protocols, security models, addressing schemes, and even alternatives to IP (Internet Protocol). On the same network, the production traffic may be isolated and processed conventionally.
- The datapath of an OpenFlow forwarding element (e.g., switch) may include a flow table, and an action associated with each flow entry included in the flow table. The set of actions supported by an OpenFlow forwarding element may be extensible. For high-performance and low-cost, the datapath may have a carefully prescribed degree of flexibility, which may mean forgoing the ability to specify arbitrary handling of each packet and seeking a more limited, but still useful, range of actions.
- An OpenFlow forwarding element may include a flow table having a plurality of flow entries (with an action associated with each flow entry) to tell the forwarding element how to process the respective flow, a secure channel that connects the switch to a remote OpenFlow controller (allowing commands and packets to be sent between the controller and the forwarding element using the OpenFlow Protocol (which provides an open and standard way for a controller to communicate with a forwarding element). By specifying a standard interface (the OpenFlow Protocol) through which entries in the forwarding element Flow Table can be defined using an external controller, researchers may not need to individually program OpenFlow forwarding elements.
- An OpenFlow forwarding element may include one or more flow tables and a group table (which may perform packet lookups and forwarding) and an OpenFlow channel to an external OpenFlow controller. The OpenFlow controller manages the forwarding element via the OpenFlow protocol. Using this protocol, the controller can add, update, and delete flow entries, both reactively (in response to packets received at the forwarding element) and proactively (e.g., to program flow tables of a new forwarding element).
- Each flow table in the forwarding element may include a set of flow entries. Each flow entry may include matched fields, counters, and a set of instructions to apply to matching packets.
- Matching at a forwarding element may start at a first flow table and may continue to additional flow tables of the forwarding element. Flow entries match data packets in priority order, with the first matching entry in each table being used. If a matching entry is found for a data packet in a flow table, the instructions associated with the specific flow entry are executed for the data packet. If no match is found for the data packet in a flow table, the outcome may depend on forwarding element configuration. The data packet may be forwarded to the controller over the OpenFlow channel, the data packet may be dropped, or attempts to match the data packet may continue to a next flow table of the forwarding element.
- Instructions associated with each flow entry describe data packet forwarding, data packet modification, group table processing, and pipeline processing. Pipeline processing instructions allow data packets to be sent to subsequent tables for further processing and allow information (e.g., in the form of metadata) to be communicated between tables. Table pipeline processing may stop when the instruction set associated with a matching flow entry does not specify a next table. At this point, the data packet may usually be modified and forwarded.
- Flow entries may forward respective data packets to a port. This is usually a physical port, but it may also be a virtual port defined by the switch or a reserved virtual port defined by the OpenFlow switch specification. Reserved virtual ports may specify generic forwarding actions such as sending to the controller, flooding, or forwarding using non-OpenFlow methods, such as “normal” switch processing, while switch-defined virtual ports may specify link aggregation groups, tunnels or loopback interfaces.
- Flow entries may also point to a group, which specifies additional processing. Groups represent sets of actions for flooding, as well as more complex forwarding semantics (e.g., multipath, fast reroute, and link aggregation). As a general layer of indirection, groups also enable multiple flows to forward to a single identifier (e.g., IP forwarding to a common next hop). This abstraction may allow common output actions across flows to be changed efficiently.
- A group table may include group entries, with each group entry including a list of action buckets with specific semantics dependent on group type. The actions in one or more action buckets are applied to data packets sent to the group.
- OpenFlow forwarding elements (e.g., switches and/or routers), controllers, and protocols are discussed, for example, in “OpenFlow Switch Specification,” Version 1.1.0 Implemented (Wire Protocol 0x02), Feb. 28, 2011, and in the reference by McKeown et al. entitled “OpenFlow: Enabling Innovation In Campus Networks,” Mar. 14, 2008. The disclosures of both of the above referenced documents are hereby incorporated herein in their entireties by reference.
- The OpenFlow channel is an interface that connects an OpenFlow forwarding element with a controller over an OpenFlow interface. The interface itself may be implementation specific, and it may be implemented using a TCP (Transmission Control Protocol) connection or a SCTP (Stream Control Transmission Protocol) connection. Moreover, TLS (Transport Layer Security) may be used to send messages that are encrypted by the controller and decrypted by the forwarding element.
- Control for a network of OpenFlow forwarding elements may be implemented using a cluster of OpenFlow controllers, and each forwarding element may use a known IP (Internet Protocol) address of one of the controllers to connect with the addressed controller according to a configuration protocol. In some implementations, a non-standard configuration channel may be used to configure a connection between a forwarding element and a respective controller. In other implementations, a forwarding element may be programmed with a list of IP addresses for controllers to connect with, and the forwarding element may sequentially attempt to connect to a controller using each controller address in the list until a successful connection with a controller is made.
- Conventionally, a connection between an OpenFlow forwarding element and an OpenFlow controller may be set up responsive to the forwarding element initiating a connection socket with the controller and requesting the connection. The controller may then decide whether to allow the connection or not. If the controller accepts the connection, the connection may be completed and messages between the controller and the forwarding element may be transmitted over the resulting OpenFlow channel. If the controller does not accept the connection, the connection socket may be terminated.
- Conventionally, once a connection is established between an OpenFlow forwarding element and an OpenFlow controller, the OpenFlow controller may only be able to drop the connection by ignoring an ECHO-REQUEST communication from the forwarding element (i.e., by not transmitting an ECHO-REPLY in response to the ECHO-REQUEST) thereby allowing the connection to timeout. Dropping a connection in this manner, however, may be dependent on a timeout value configuration of the OpenFlow Request-Reply protocol. Stated in other words, once the controller decides to drop a connection with a forwarding element, the connection may not actually be dropped until after a next ECHO-REQUEST has been transmitted by the forwarding element and a timeout period has passed after the ECHO-REQUEST without transmitting an ECHO-REPLY.
- Moreover, conventional mechanisms to connect an OpenFlow forwarding element with an OpenFlow switch may be limited. A complex configuration channel may be adopted, for example, to configure an OpenFlow forwarding element to connect with a controller from a list of controllers known to the forwarding element. Using such a known list of controllers, however, may make it difficult for a forwarding element to connect to an unknown controller (e.g., a controller added to the cluster after provisioning the list). In addition, it may be difficult to share the load of network forwarding elements in the forwarding plane among controllers in the control plane.
- In some controller implementations, one controller of a cluster of controllers may be designated as a master controller of the cluster, and only the master controller may be allowed to accept OpenFlow connections from network forwarding elements. If another controller is later designated among the controllers as the master controller or if the master controller fails in a conventional arrangement, the forwarding element may be delayed in connecting to a new master controller and/or the forwarding element may be unable to connect to the new master element.
- Accordingly, there continues to exist a need in the art for improved operations in networks including forwarding elements and controllers.
- It is therefore an object to address at least some of the above mentioned disadvantages and/or to improve network performance.
- According to some embodiments, a method of connecting a network forwarding element may include transmitting a first connection request from the forwarding element to a first network controller. After transmitting the first connection request, a rejection message may be received from the first network controller at the network forwarding element, with the rejection message being responsive to the first connection request and with the rejection message including an address of a second network controller. Responsive to receiving the rejection message, a second connection request may be transmitted from the network forwarding element to the second network controller.
- By allowing a controller to redirect a forwarding element to another controller, a forwarding element may be efficiently connected to a desired controller even through the desired controller is not known to the forwarding element at the time of the connection request. Such redirection may thus facilitate connection of a new forwarding element to an appropriate controller in a network, connection of a forwarding element to a new master controller after loss of a previous master controller, and/or distribution of forwarding element loads across a plurality of peer controllers.
- The rejection message may include the address of the second network controller and an address of a third network controller. Moreover, the first and second network controllers may be respective first and second OpenFlow network controllers, the network forwarding element may be an OpenFlow network forwarding element, and the rejection message may be transmitted over an OpenFlow channel.
- The network forwarding element may include a flow table having a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream. In addition, an acceptance message may be received from the second network controller at the network forwarding element after transmitting the second connection request, and an instruction to change at least one of the plurality of flow entries may be received from the second network controller at the network forwarding element after receiving the acceptance message. Moreover, the at least one of the plurality of flow entries may be changed responsive to the instruction received from the second network controller. After changing the at least one of the plurality of flow entries, a data packet of a data stream may be processed through the network forwarding element in accordance with the at least one of the plurality of flow entries including the change.
- The rejection message may include the address of the second network controller and an address of a third network controller, and a third connection request may be transmitted from the network forwarding element to the third network controller responsive to transmitting the second connection request without establishing a connection with the second network controller.
- The network forwarding element may include a flow table having a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream. After transmitting the third connection request, an acceptance message may be received from the third network controller at the network forwarding element. After receiving the acceptance message, an instruction to change at least one of the plurality of flow entries may be received from the third network controller at the network forwarding element. Moreover, the at least one of the plurality of flow entries may be changed responsive to the instruction received from the third network controller.
- According to some other embodiments, a method of redirecting a network forwarding element may include receiving a connection request at a first network controller from the network forwarding element. Responsive to receiving the connection request, the controller may determine whether to accept control of the forwarding element at the first network controller. Responsive to determining to not accept control of the forwarding element at the first network controller, a rejection message may be transmitted from the first network controller to the network forwarding element, with the rejection message including an address of a second network controller.
- The rejection message may include the address of the second network controller and an address of a third network controller. Moreover, the first and second network controllers may be respective first and second OpenFlow network controllers, the network forwarding element may be an OpenFlow network forwarding element, and the rejection message may be transmitted over an OpenFlow channel.
- The network forwarding element may include a flow table having a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream. Responsive to determining to accept control of the forwarding element at the first network controller, an acceptance message may be transmitted from the first network controller to the network forwarding element. After transmitting the acceptance message, an instruction to change at least one of the plurality of flow entries may be transmitted from the first network controller to the network forwarding element.
- According to still other embodiments, a network forwarding element may include a channel interface configured to provide communications between the network forwarding element in a forwarding plane and a control plane including a plurality of controllers, and a processor coupled to the channel interface. The processor may be configured to transmit a first connection request through the channel interface to a first network controller of the control plane, to receive a rejection message from the first network controller through the channel interface after transmitting the first connection request. The rejection message may be responsive to the first connection request and the rejection message may include an address of a second network controller of the control plane. In addition, the processor may be configured to transmit a second connection request from the network forwarding element to the second network controller responsive to receiving the rejection message.
- The rejection message may include the address of the second network controller and an address of a third network controller. The first and second network controllers may be respective first and second OpenFlow network controllers, the network forwarding element may be an OpenFlow network forwarding element, the channel interface may be an OpenFlow secure channel interface, and the rejection message may be transmitted over an OpenFlow channel.
- The network forwarding element may further include a plurality of input ports configured to receive data packets of data streams from other forwarding elements, and a plurality of output ports configured to transmit the data packets of the data streams to other forwarding elements, with the processor being coupled between the input and output ports. A flow table may be coupled to the processor, with the flow table including a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream between respective ones of the input and output ports, between a respective one of the input ports and the control plane, and/or between the control plane and a respective one of the output ports. Moreover, the processor may be further configured to receive an acceptance message from the second network controller through the channel interface after transmitting the second connection request, to receive an instruction to change at least one of the plurality of flow entries from the second network controller through the channel interface after receiving the acceptance message, and to change the at least one of the plurality of flow entries of the flow table responsive to the instruction received from the second network controller.
- The processor may be further configured to process a data packet of a data stream between respective ones of the input and output ports, between respective ones of the input ports and the control plane, and/or between the control plane and respective ones of the output ports in accordance with the at least one of the plurality of flow entries including the change.
- The rejection message may include the address of the second network controller and an address of a third network controller, with the processor being further configured to transmit a third connection request through the channel interface to the third network controller responsive to transmitting the second connection request without establishing a connection with the second network controller.
- The forwarding element may further include a plurality of input ports configured to receive data packets of data streams from other forwarding elements and a plurality of output ports configured to transmit the data packets of the data streams to other forwarding elements, with the processor being coupled between the input and output ports. In addition, a flow table may be coupled to the processor, with the flow table including a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream between respective ones of the input and output ports, between a respective one of the input ports and the control plane, and/or between the control plane and a respective one of the output ports. The processor may be further configured to receive an acceptance message from the third network controller through the channel interface after transmitting the third connection request, to receive an instruction to change at least one of the plurality of flow entries from the third network controller through the channel interface after receiving the acceptance message, and to change the at least one of the plurality of flow entries responsive to the instruction received from the third network controller.
- According to yet further embodiments, a network controller may include a channel interface configured to provide communications between the network controller in a control plane and a forwarding plane including a plurality of network forwarding element, and a processor coupled to the channel interface. The processor may be configured to receive a connection request through the channel interface from the network forwarding element, to determine whether to accept control of the forwarding element at the first network controller responsive to receiving the connection request, and to transmit a rejection message through the channel interface to the network forwarding element responsive to determining to not accept control of the forwarding element, with the rejection message including an address of a second network controller.
- The rejection message may include the address of the second network controller and an address of a third network controller. The first and second network controllers may be respective first and second OpenFlow network controllers, the network forwarding element may be an OpenFlow network forwarding element, the channel interface may be an OpenFlow secure channel interface, and the rejection message may be transmitted over an OpenFlow channel.
- The network forwarding element may include a flow table having a plurality of flow entries with each of the plurality of flow entries defining processing for data packets of a respective data stream. Moreover the processor may be further configured to transmit an acceptance message through the channel interface to the network forwarding element responsive to determining to accept control of the forwarding element, and to transmit an instruction to change at least one of the plurality of flow entries through the channel interface to the network forwarding element after transmitting the acceptance message.
- The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiment(s) of the invention. In the drawings:
-
FIG. 1 is a block diagram illustrating a network including controllers and forwarding elements according to some embodiments; -
FIG. 2 is a block diagram illustrating a forwarding element ofFIG. 1 according to some embodiments; -
FIG. 3 is a block diagram illustrating a controller ofFIG. 1 according to some embodiments; -
FIG. 4 is a flow chart illustrating operations of a forwarding element according to some embodiments; -
FIG. 5 is flow chart illustrating operations of a controller according to some embodiment; and -
FIG. 6 is a flow diagram illustrating communications between a forwarding element and controllers according to some embodiments. - Embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which examples of embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present/used in one or more other embodiments.
- According to some embodiments discussed herein, the OpenFlow protocol may be extended to provide redirection of connection between an OpenFlow forwarding element in a network forwarding plane from one OpenFlow controller to another OpenFlow controller in a network control plane. Communications may be provided between the controllers so that each controller may be aware of the existence and/or status of the other controllers in the control plane. Accordingly, any controller in the control plane that receives a connection request from a forwarding element in the forwarding plane may redirect the forwarding element to another controller that may be more appropriate for connection with the forwarding element.
-
FIG. 1 is a block diagram illustrating a network including a plurality of OpenFlow controllers 101-1 to 101-n (controllers) in a control plane and a plurality of OpenFlow forwarding elements 103-1 to 103-m (forwarding elements) in a forwarding plane, with communications between OpenFlow Controllers 101-1 to 101-n and OpenFlow forwarding elements 103-1 to 103-m being provided though an OpenFlow channel (a channel) therebetween in accordance with an OpenFlow protocol(s). The OpenFlow channel may be implemented, for example, using a Transmission Control Protocol (TCP) connection and/or a Stream Control Transmission Protocol (SCTP) connection, and Transport Layer Security (TLS) may be used to communicate encrypted messages between respective controllers and forwarding elements. Moreover, communication paths 111 (separate from the OpenFlow channel) may be provided between OpenFlow forwarding elements 103-1 to 103-m in the forwarding plane to support transmission of data packets between forwarding elements. In addition, communication paths 115 (separate from the OpenFlow channel) may be provided between controllers 101-1 to 101-n in the control plane to support coordination between controllers. -
FIG. 2 is a block diagram illustrating elements of anOpenFlow forwarding element 103 ofFIG. 1 . As shown, each forwardingelement 103 may include aprocessor 207 coupled between one ormore input ports 205 and one ormore output ports 209. One or more flow tables 201 may be coupled withprocessor 207, each flow table may include a plurality of flow entries, and each flow entry may define instructions used to process/forward data packets matching the respective flow entry. Stated in other words, each flow entry may define processing for data packets of a respective data stream(s). Data packets from other forwarding elements may be received atrespective input ports 205, processed byprocessor 207 in accordance with respective flow entries from flow table(s) 201, and transmitted fromrespective output ports 209 to other forwarding elements. - Each forwarding
element 103 may also includememory 211, group tables 215, and OpenFlow Secure Channel Interface 203 (a channel interface). More particularly, OpenFlowsecure channel interface 203 may provide communications betweenprocessor 207 and a respectiveOpenFlow controller 101 through the OpenFlow Channel ofFIG. 1 . OpenFlowsecure channel interface 203, for example, may use TLS to encrypt/send messages over the OpenFlow Channel to a respective controller and/or to receive/decrypt messages transmitted by the respective controller over the OpenFlow Channel. In the event that a data packet is received through one ofinput ports 205 andprocessor 207 cannot match the data packet with a flow entry in flow table(s) 201, for example,processor 207 may transmit the data packet through OpenFlowSecure Channel Interface 203 over the OpenFlow Channel to a respective controller. In the other direction,processor 207 may receive messages from the respective controller over the OpenFlow Channel throughOpenFlow Channel interface 203 to change (e.g., add, update, and/or delete) a flow entry(ies) of flow table(s) 201. -
FIG. 3 is a block diagram illustrating elements of anOpenFlow controller 101 ofFIG. 1 . As shown, eachcontroller 101 may include aprocessor 307 coupled between OpenFlow Secure Channel Interface 303 (channel interface) andcontroller interface 309. OpenFlowSecure Channel Interface 303 may provide communications betweenprocessor 307 and a respectiveOpenFlow forwarding element 103 through the OpenFlow Channel ofFIG. 1 . OpenFlowsecure channel interface 303, for example, may use TLS to encrypt/send messages over the OpenFlow Channel to a respective forwarding element(s) and/or to receive/decrypt messages transmitted by the respective forwarding element(s) over the OpenFlow Channel.OpenFlow controller 101 may also includememory 311 coupled toprocessor 307. - Operations of OpenFlow forwarding element 103 (a forwarding element) and OpenFlow controller 101 (a controller) of
FIGS. 1 , 2, and 3 will be discussed in greater detail below with respect to the flow charts ofFIGS. 4 and 5 . Because operations of forwardingelement 103 andcontroller 101 of embodiments discussed herein are interrelated, operations ofFIGS. 4 and 5 will be discussed together. Moreover, each forwarding element ofFIG. 1 may have a structure as discussed above with respectFIG. 2 , and each controller ofFIG. 1 may have a structure as discussed above with respect toFIG. 3 . - As discussed above, communications between controllers 101-1 to 101-n may be provided through
respective controller interfaces 309 of each controller. Accordingly, each controller may maintain inmemory 311 thereof a listing of addresses for all of the other controllers of the control plane, and this listing of addresses and/or other network information may be used by each controller to identify one or more other controllers to which forwarding elements may be redirected. - Before forwarding
element 103 is coupled into the forwarding plane of the network ofFIG. 1 , forwardingelement 103memory 211 may be programmed with an address of a controller or a list of addresses of controllers with which it should initiate connection when coupled to the network. The controller address or addresses, for example, may be programmed during manufacturing and/or later using a default controller address or addresses. More particularly, the controller address(es) may be an Internet Protocol (IP) address(es). - When forwarding element 103-1, for example, is initially coupled to the network of
FIG. 1 ,processor 207 may detect that OpenFlow secure channel interface 203 (channel interface) is coupled to the OpenFlow channel.Processor 207 detecting communication with the control plane may serve as a triggering event forprocessor 207 to initiate connection to a controller of the control plane atblock 401 ofFIG. 4 . While initial connection to the OpenFlow channel is discussed as an example of a triggering event, other events may trigger connection/reconnection atblock 401. For example, loss of an existing connection with a controller may trigger connection/reconnection atblock 401. - In response to the triggering event (e.g., detecting communication with the OpenFlow Channel and/or loss of an existing connection with a controller),
processor 207 may identify a controller address (e.g., an IP address of a controller) frommemory 211 for network connection atblock 403 ofFIG. 4 . As discussed above, one controller address or a list of controller addresses may be saved inmemory 211. The only controller address or the first controller address of a list may be selected byprocessor 207 for network connection. - At
block 405,processor 207 may transmit a connection request using the controller address selected frommemory 211. For purposes of this discussion, the controller address may identify controller 101-1 ofFIG. 1 . For example,processor 207 may initiate a socket connection betweenchannel interface 203 of forwarding element 103-1 andchannel interface 303 of controller 101-1 over the OpenFlow channel using the address of controller 101-1 obtained frommemory 211, and the connection request may be transmitted to controller 101-1 using the socket connection. If the socket connection cannot be established and a list of controller addresses is programmed inmemory 211,processor 207 may attempt to establish a socket connection sequentially using each controller address of the list until a socket connection is established, and the communication request may be transmitted to the controller using the socket connection atblock 405. - When the connection request is received at
processor 307 of controller 101-1 (through the OpenFlow channel and channel interface 303) atblock 501 ofFIG. 5 , processor 307 (of controller 101-1) may decide atblock 503 ofFIG. 5 whether to accept or reject the connection request from forwarding element 103-1. Ifprocessor 307 accepts the connection request atblock 503,processor 307 may transmit an acceptance message atblock 507 throughchannel interface 303 over the OpenFlow channel to forwarding element 103-1. Responsive to receiving the acceptance message over the OpenFlow channel and throughchannel interface 203 atblock 407,processor 207 of forwarding element 103-1 may proceed with operations ofblocks 421 to 427 ofFIG. 4 , and after transmitting the acceptance message,processor 307 of controller 101-1 may proceed withoperations 509 to 515 ofFIG. 5 . - At
block 509 ofFIG. 5 ,processor 307 of controller 101-1 may determine if flow entries of flow table(s) 201 of forwarding element 103-1 should be changed (e.g., added, updated, and/or deleted).Processor 307 of controller 101-1 may decide to add one or more new flow entries, update one or more existing flow entries, and/or or delete one or more existing flow entries of flow table(s) 201 of forwarding element 103-1. Such changes may be made proactively (e.g., to initially populate one or more flow tables, to balance loads across multiple forwarding elements, etc.) or reactively (e.g., to accommodate new data streams responsive to unmatched data packets at forwarding element 103-1). Ifprocessor 307 decides to change a flow entry atblock 509,processor 307 may transmit a change instruction(s) throughchannel interface 303 over OpenFlow channel to forwarding element 103-1 atblock 511. Moreover, operations ofblocks block 515. - If
processor 207 of forwarding element 103-1 receives such a change instruction(s) from controller 101-1 over OpenFlow channel throughchannel interface 203 atblock 421,processor 207 may change (e.g., add, update, and/or delete) at least one flow entry of its flow table(s) responsive to the change instructions atblock 423, andprocessor 207 may process data packets through forwarding element 103-1 in accordance with the flow tables (including any changes implemented at block 423) atblock 425. Moreover, operations ofblocks block 427. - During the course of processing data packets through forwarding element 103-1 at
block 425, one or more data packets may not match any flow entries of flow table(s) 201, in which case,processor 307 may forward the non-matching data packet(s) throughchannel interface 203 and over OpenFlow channel to controller 101-1. On receipt of the non-matching data packet over the OpenFlow channel and throughchannel interface 303,processor 307 of controller 101-1 may decide to add or update a flow entry of flow table(s) 201 of forwarding element 103-1 to handle a new data stream including the non-matching data packet.Processor 307 of controller 101-1 may transmit the instruction atblock 511,processor 207 of forwarding element 103-1 may receive the instruction atblock 421, andprocessor 207 of forwarding element 103-1 may add/update a flow entry of flow table(s) 201 in accordance with the instruction.Processor 207 of forwarding element 103-1 may then process subsequent data packets of the new data stream in accordance with the added/updated flow entry. - As discussed above, if
processor 307 of controller 101-1 accepts the connection request atblock 503 and transmits the acceptance message to forwarding element 103-1 atblock 507, forwarding element 103-1 and controller 101-1 may proceed withoperations FIG. 4 and withoperations FIG. 5 . In the alternative,processor 307 of controller 101-1 may reject the connection request atblock 503 and transmit a rejection message throughchannel interface 303 over the OpenFlow channel to forwarding element 103-1 atblock 505 ofFIG. 5 . More particularly, the rejection message may include an address (e.g., an IP address) of a secondary controller (e.g., controller 101-2) or a list of addresses (e.g., IP addresses) of respective secondary controllers (e.g., controllers 101-2, 101-3, 101-4, etc.). The rejection message, for example, may include an address/addresses for one or more secondary controllers that may be more suitable to control forwarding element 103-1. On transmission/reception of the rejection message, the socket connection between forwarding element 103-1 and controller 101-1 may be dropped. - On receipt of the rejection message at
block 407 over the OpenFlow channel throughchannel interface 203,processor 207 of forwarding element 103-1 may transmit a second connection request (throughchannel interface 203 over the OpenFlow channel) to a secondary controller (e.g., controller 101-2) using the first address from the list of addresses of secondary controllers if a list is provided (or using the only address if only one address is provided) atblock 411 ofFIG. 4 . More particularly,processor 207 may establish a socket connection betweenchannel interface 203 of forwarding element 103-1 andchannel interface 303 of secondary controller 101-2 over the OpenFlow channel using the address of controller 101-2, and the second connection request may be transmitted over the socket connection. - The secondary controller (e.g., controller 101-2) may then proceed with operations of
FIG. 5 as discussed above with respect to primary controller 101-1. On receipt of the connection request (over the OpenFlow channel throughchannel interface 303 of secondary controller 101-2) atblock 501,processor 307 of secondary controller 101-2 may either accept or reject the connection request atblock 503, and either transmit an acceptance message atblock 507 or a rejection message atblock 505. - If
processor 307 of controller 101-2 transmits an acceptance message atblock 507 and the acceptance message is received byprocessor 207 of forwarding element 103-1 atblock 415,processor 207 of forwarding element 103-1 may proceed with operations ofblocks processor 307 of controller 101-2 may proceed with operations ofblocks - If
processor 307 of controller 101-2 transmits a rejection message atblock 505, the rejection message may include a secondary controller address or a list of secondary controller addresses to be used byprocessor 207 of forwarding element 103-1 for subsequent connection requests. In an alternative, a rejection message from a secondary controller may omit addresses of other secondary controllers so that the forwarding element 103-1 continues using secondary controller addresses provided in a list from the first rejection message from primary controller 101-1. Upon transmission of the rejection message atblock 505 and/or receipt of the rejection message atblock 415, the socket connection between forwarding element 103-1 and controller 101-2 may be dropped. - If a rejection message is received by
processor 207 from secondary controller 101-2 over the OpenFlow channel throughchannel interface 203 atblock 415,processor 207 of forwarding element 103-1 may determine if all addresses of secondary controllers have been exhausted atblock 417.Processor 207 of forwarding element 103-1 may thus repeat operations ofblocks block 407 from primary controller 101-1 may include a list of addresses of secondary controllers, andprocessor 207 of forwarding element 103-1 may proceed through operations ofblocks block 415 or all of the addresses of the list have been exhausted atblock 417. According to some other embodiments, each rejection message received atblock 415 may include an address(es) of a secondary controller(s) that is used for a subsequent connection request atblock 411. -
FIG. 6 is a message flow diagram illustrating an example of communications between forwarding element 101-1 and controllers 101-1, 101-2, and 101-3 over the OpenFlow channel as shown inFIG. 1 according to operations ofFIGS. 4 and 5 . Forwarding element 103-1 may have the structure illustrated inFIG. 2 and may operate in accordance with operations of the flow chart ofFIG. 4 . Each of controllers 101-1, 101-2, and 101-3 may separately have the structure ofFIG. 3 and may operate independently in accordance with operations of the flow chart ofFIG. 5 . Moreover, operations from the flow charts ofFIGS. 4 and 5 are provided in parenthesis inFIG. 6 to illustrate a correspondence of messages ofFIG. 6 and operations ofFIGS. 4 and 5 . - As shown in
FIG. 4 , forwarding element 103-1 may transmit a first connection request to controller 101-1 (acting as a primary controller) and the connection request may be received at controller 101-1 as discussed above with respect tooperations FIGS. 4 and 5 . Forwarding element 103-1, for example, may transmit the first connection request responsive to detecting initial connection to the OpenFlow channel or responsive to loss of a previous connection with another controller. - Responsive to receiving the first connection request and rejecting the connection request, controller 101-1 may transmit a rejection message to forwarding element 103-1 and the rejection message may be received at forwarding element 103-1 as discussed above with respect to
operations FIGS. 4 and 5 . Moreover, the rejection message may include addresses of first and second secondary controllers 101-2 and 101-3 to be used by forwarding element 103-1 for subsequent connection requests. - Upon receipt of the rejection message including the addresses of first and second secondary controllers 101-2 and 101-3, forwarding element 103-1 may transmit a second connection request to first secondary controller 101-2 using the first address in the list provided with the rejection message as discussed above with respect to
operation 411 ofFIG. 4 . In this example, first secondary controller 101-2 may be out of service so that the second connection request is not received by the second controller and/or no acceptance or rejection message is transmitted by first secondary controller 101-2. Accordingly, no acceptance or rejection message is received by forwarding element 103-1 atblock 415. In this case, failure to receive a response to the second connection request within a designated timeout period may be interpreted as a rejection atblock 415 ofFIG. 4 . - Accordingly, forwarding element 103-1 may transmit a third connection request to second secondary controller 101-3 using the second address in the list provided with the rejection message from controller 101-1 as discussed above with respect to
operation 411 ofFIG. 4 , and the third connection request may be received at the second secondary controller 101-3 as discussed above with respect tooperation 501 ofFIG. 5 . - Responsive to receiving the third connection request and accepting the third connection request, controller 101-3 may transmit an acceptance message to forwarding element 103-1 and the acceptance message may be received at forwarding element 103-1 as discussed above with respect to
operations FIGS. 4 and 5 . OpenFlow communications may then be provide over the OpenFlow channel between forwarding element 103-1 and controller 101-3 as discussed above with respect tooperations FIG. 5 andoperations FIG. 4 . - The rejection message of
operation 505 ofFIGS. 5 and 6 (also referred to as a redirection message) may be provided as a new message type, such as an OFPT_CONNECTION_REJECT message having the following format: -
DEFINITIONS OF REJECTION MESSAGE FORMAT MESSAGE ELEMENTS struct ofpt_connection_reject { message name struct ofpt_header header; header u_int32_t num_of_controllers; number of secondary controller addresses u_int32_t ofpt_controller_ip_list[ ]; list of secondary controller addresses };
The header (struct ofpt_header header) may be an OpenFlow header including the version, type, length, and xid (a unique identification for the message), and a transaction identification. The transaction identification may be required to match a reply from the forwarding element with a request. With a rejection message discussed herein, a transaction identification may not be required because a forwarding element reply may not be received for a rejection message. The “num_of controllers” field may indicate a number of controller IP addresses included in the rejection message, and the “ofpt_controller_ip_list” provides the list of secondary controller IP addresses. - An initial connection request (e.g., an OFPT_CONNECTION_REJECT message) may be transmitted from forwarding element 103-1 at
block 405 ofFIG. 4 and received by controller 101-1 atblock 501 ofFIG. 5 , and controller 101-1 may then decide whether to accept the connection request as discussed above with respect to block 503 ofFIG. 5 . If controller 101-1 decides to accept the connection request, controller 101-1 may respond with an OFPT_FEATURES_REQUEST message as an acceptance message, forwarding element 103-1 may respond with an OFPT_FEATURES_REPLY message, and then a finite state machine (FSM) of controller 101-1 may progress normally to complete the connection between forwarding element 103-1 and controller 101-1. - If controller 101-1 decides to reject a connection request, controller 101-1 may transmit a rejection message as discussed above with respect to
operation 505 ofFIG. 5 . More particularly, the rejection message may be provided as an OFPT_CONNECTION_REJECT message discussed above, with the “ofpt_controller_ip_list” field including a list of one or more IP addresses of secondary controllers (e.g., IP addresses for controllers 101-2, 101-3, 101-4, etc.), with the “num_of_controllers” field set to indicate the number of IP addresses included in the list, and with the message type set to “OFPT_CONNECTION_REJECT.” Controller 101-1 may then send the rejection message to forwarding element 101-1 as discussed above with respect to block 505 ofFIG. 5 and issue a connection shut down to shut down the socket connection between forwarding element 103-1 and controller 101-1. The forwarding element 103-1 may also close the socket connection, and then use the controller IP addresses from the “ofpt_controller_ip_list” field for subsequent connection attempts in accordance withoperations FIG. 4 . Forwarding element 101-1 may also save the controller IP addresses from the “ofpt_controller_ip_list” field for use in the event that a subsequent connection with a secondary controller is lost. - According to some embodiments, a cluster of controllers 101-1 to 101-n may be provided in the control plane, and one of the controllers (e.g., controller 101-1) may be elected as a master of the cluster. The remaining controllers (e.g., controllers 101-2 to 101-n) may be stand-by or secondary controllers that are ready to take over in the event that master controller 101-1 fails. Accordingly, all forwarding elements 103-1 to 103-m in the forwarding plane may be initially connected to master controller 101-1.
- In the event that master controller 101-1 fails, the secondary controllers 101-2 to 101-n may need to elect a new master controller, and all forwarding elements 103-1 to 103-n may need to reconnect with the new master controller. Remaining controllers 101-2 to 101-n may elect a new master (e.g., controller 101-2) based on information obtained/provided over
communication paths 115, but forwarding elements 103-1 to 103-m may be unaware of this election. Accordingly, one or more of forwarding elements 103-1 to 103-m may attempt reconnection with the wrong controller. - By way of example, original master controller 101-1 may fail, and the remaining controllers 101-2 to 101-n of the control plane may elect controller 101-2 as the new master controller. Upon failure to receive respective ECHO_REPLY messages from original master controller 101-1 within a timeout period, each of forwarding elements 103-1 to 103-m may attempt to connect with one of the other controllers 101-2 to 101-m without knowing that controller 101-2 has been elected as the new master controller. Accordingly, some of forwarding elements 103-1 to 103-m may connect with one of non-master controllers 101-3 to 101-n. In this situation, the non-master controllers 101-3 to 101-n may use rejection messages (also referred to as redirection messages) to redirect forwarding elements 103-1 to 103-m to the new master controller 101-2.
- By way of example, all forwarding elements 103-1 to 103-n may be initially connected to original master controller 101-1 when master controller fails. Upon timeouts after respective failure to receive respective ECHO_REPLY messages from original master controller 101-1, each forwarding element 103-1 to 103-n may attempt to connect with another controller in accordance with operations discussed above with respect to
FIG. 4 . - Some of the forwarding elements may correctly initiate connection with new master controller 101-2, but others may initiate connection with other controllers. Forwarding element 103-1, for example, may initiate connection with controller 101-3. In this case, forwarding element 103-1 and controller 101-3 may follow
operations FIG. 4 andoperations FIG. 5 to redirect forwarding element 103-1 to new master controller 101-2. Stated in other words, controller 101-3 may transmit a rejection message atblock 505 ofFIG. 5 including an IP address of the new master controller 101-1. Accordingly, forwarding element 103-1 may proceed with operation of 411, 415, and 417 ofFIG. 4 to connect with the new master controller 101-2. Similarly, any of controllers 101-3 to 101-n may use a rejection message including the IP address of the new master controller 101-2 to efficiently redirect forwarding elements to the new master controller. - According to other embodiments, no one controller may be designated as a mater controller, and instead, all controllers 101-1 to 101-n in the control plane may be peer controllers, with each forwarding element being connected to one of the peer controllers. Accordingly, peer controllers 101-1 to 101-n may share the load of forwarding elements 103-1 to 103-m. Moreover, controllers 101-1 to 101-n may share data/state information over
communication paths 115. - When one of the forwarding elements (e.g., forwarding element 103-1) attempts connection with one of the controllers (e.g., controller 101-1), the controller may determine whether to accept or reject the connection request based on its load and/or based on loads of other controllers. If the controller determines that the connection with the forwarding element is best handled by another controller (e.g., because its load is relatively high and/or because a load of another controller is relatively low), the controller may transmit a rejection message including an IP address of another controller (e.g., another controller that is currently experiencing a lower load). Rejection messages may thus be used to share forwarding element loads across controllers of the control plane.
- According to embodiments discussed above, a plurality of controllers may be deployed in the control plane, and rejection messages may be used to control/redirect connections between forwarding elements of the forwarding plane and controllers of the control plane. Moreover, new forwarding elements and/or controllers may be deployed in the forwarding and control planes while maintaining network operations without requiring an elaborate configuration protocol. In addition, rejection messages may be used to manage maintenance of the network.
- In the above-description of various embodiments of the present invention, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense expressly so defined herein.
- When an element is referred to as being “connected”, “coupled”, “responsive”, or variants thereof to another element, it can be directly connected, coupled, or responsive to the other element or intervening elements may be present. In contrast, when an element is referred to as being “directly connected”, “directly coupled”, “directly responsive”, or variants thereof to another element, there are no intervening elements present. Like numbers refer to like elements throughout. Furthermore, “coupled”, “connected”, “responsive”, or variants thereof as used herein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and/or clarity. The term “and/or” includes any and all combinations of one or more of the associated listed items.
- As used herein, the terms “comprise”, “comprising”, “comprises”, “include”, “including”, “includes”, “have”, “has”, “having”, or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof. Furthermore, as used herein, the common abbreviation “e.g.”, which derives from the Latin phrase “exempli gratia,” may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item. The common abbreviation “i.e.”, which derives from the Latin phrase “id est,” may be used to specify a particular item from a more general recitation.
- Example embodiments are described herein with reference to block diagrams and/or flowchart illustrations of computer-implemented methods, apparatus (systems and/or devices) and/or computer program products. It is understood that a block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and/or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and/or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions/acts specified in the block diagrams and/or flowchart block or blocks, and thereby create means (functionality) and/or structure for implementing the functions/acts specified in the block diagrams and/or flowchart block(s).
- These computer program instructions may also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions/acts specified in the block diagrams and/or flowchart block or blocks.
- A tangible, non-transitory computer-readable medium may include an electronic, magnetic, optical, electromagnetic, or semiconductor data storage system, apparatus, or device. More specific examples of the computer-readable medium would include the following: a portable computer diskette, a random access memory (RAM) circuit, a read-only memory (ROM) circuit, an erasable programmable read-only memory (EPROM or Flash memory) circuit, a portable compact disc read-only memory (CD-ROM), and a portable digital video disc read-only memory (DVD/BlueRay).
- The computer program instructions may also be loaded onto a computer and/or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer and/or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the block diagrams and/or flowchart block or blocks. Accordingly, embodiments of the present invention may be embodied in hardware and/or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as “circuitry,” “a module” or variants thereof.
- It should also be noted that in some alternate implementations, the functions/acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Moreover, the functionality of a given block of the flowcharts and/or block diagrams may be separated into multiple blocks and/or the functionality of two or more blocks of the flowcharts and/or block diagrams may be at least partially integrated. Finally, other blocks may be added/inserted between the blocks that are illustrated. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.
- Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate every combination and subcombination of these embodiments. Accordingly, the present specification, including the drawings, shall be construed to constitute a complete written description of various example combinations and subcombinations of embodiments and of the manner and process of making and using them, and shall support claims to any such combination or subcombination.
- Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present invention. All such variations and modifications are intended to be included herein within the scope of the present invention.
Claims (22)
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/369,993 US9130869B2 (en) | 2012-02-09 | 2012-02-09 | Methods of redirecting network forwarding elements and related forwarding elements and controllers |
PCT/IB2013/050946 WO2013118049A1 (en) | 2012-02-09 | 2013-02-04 | Methods of redirecting network forwarding elements and related forwarding elements and controllers |
IN6960DEN2014 IN2014DN06960A (en) | 2012-02-09 | 2013-02-04 | |
EP13713971.3A EP2813034B1 (en) | 2012-02-09 | 2013-02-04 | Methods of redirecting network forwarding elements and related forwarding elements and controllers |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/369,993 US9130869B2 (en) | 2012-02-09 | 2012-02-09 | Methods of redirecting network forwarding elements and related forwarding elements and controllers |
Publications (2)
Publication Number | Publication Date |
---|---|
US20130208621A1 true US20130208621A1 (en) | 2013-08-15 |
US9130869B2 US9130869B2 (en) | 2015-09-08 |
Family
ID=48045606
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/369,993 Active 2034-05-12 US9130869B2 (en) | 2012-02-09 | 2012-02-09 | Methods of redirecting network forwarding elements and related forwarding elements and controllers |
Country Status (4)
Country | Link |
---|---|
US (1) | US9130869B2 (en) |
EP (1) | EP2813034B1 (en) |
IN (1) | IN2014DN06960A (en) |
WO (1) | WO2013118049A1 (en) |
Cited By (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130223440A1 (en) * | 2012-02-28 | 2013-08-29 | International Business Machines Corporation | Disjoint multi-pathing for a data center network |
US20140089545A1 (en) * | 2012-09-21 | 2014-03-27 | Lsi Corporation | Leased lock in active-active high availability das systems |
US20140108632A1 (en) * | 2012-10-15 | 2014-04-17 | Cisco Technology, Inc. | System and method for efficient use of flow table space in a network environment |
US20140169158A1 (en) * | 2012-12-17 | 2014-06-19 | Telefonaktiebolaget L M Ericsson (Publ) | Extending the reach and effectiveness of header compression in access networks using sdn |
US8908539B1 (en) * | 2012-05-25 | 2014-12-09 | Google Inc. | Systems and methods for testing network connections of a centrally-controlled network |
US20140369259A1 (en) * | 2013-06-14 | 2014-12-18 | Samsung Electronics Co., Ltd. | Apparatus and method for transmitting/receiving data in mobile content network |
US20150117458A1 (en) * | 2013-10-24 | 2015-04-30 | University Of Houston System | Location-based network routing |
JP2015104070A (en) * | 2013-11-27 | 2015-06-04 | エヌ・ティ・ティ・コムウェア株式会社 | Network management system, management device, controller, management method, control method, and program |
CN104717153A (en) * | 2013-12-12 | 2015-06-17 | 国际商业机器公司 | Resource over-subscription |
US20150319062A1 (en) * | 2014-05-05 | 2015-11-05 | Nicira, Inc. | Buffered subscriber tables for maintaining a consistent network state |
US20150350156A1 (en) * | 2012-12-26 | 2015-12-03 | Zte Corporation | NAT implementation system, method, and Openflow switch |
US20160028623A1 (en) * | 2012-04-10 | 2016-01-28 | International Business Machines Corporation | Switch routing table utilizing software defined network (sdn) controller programmed route segregation and prioritization |
US9350677B2 (en) | 2014-01-16 | 2016-05-24 | International Business Machines Corporation | Controller based network resource management |
CN105991606A (en) * | 2015-02-27 | 2016-10-05 | 中兴通讯股份有限公司 | OpenFlow message processing method and network element |
US20160294776A1 (en) * | 2015-04-06 | 2016-10-06 | Nicira, Inc. | Security cluster for performing security check |
CN106357450A (en) * | 2016-09-29 | 2017-01-25 | 全球能源互联网研究院 | Method for mapping asynchronous information of OpenFlow protocol |
WO2017071743A1 (en) * | 2015-10-28 | 2017-05-04 | Huawei Technologies Co., Ltd. | Control traffic in software defined networks |
US20180081919A1 (en) * | 2014-09-02 | 2018-03-22 | Ab Initio Technology Llc | Compiling graph-based program specifications |
US10228976B2 (en) * | 2012-05-01 | 2019-03-12 | Keysight Technologies Singapore (Holdings) Pte. Ltd. | Methods, systems, and computer readable media for balancing incoming connections across multiple cores |
US20200236054A1 (en) * | 2015-06-30 | 2020-07-23 | Huawei Technologies Co., Ltd. | Routing method, related device, and system |
US10749801B2 (en) * | 2016-06-29 | 2020-08-18 | Nicira, Inc. | Installation of routing tables for logical router in route server mode |
US10795716B2 (en) | 2015-10-31 | 2020-10-06 | Nicira, Inc. | Static route types for logical routers |
US10797998B2 (en) | 2018-12-05 | 2020-10-06 | Vmware, Inc. | Route server for distributed routers using hierarchical routing protocol |
US10805212B2 (en) | 2015-08-11 | 2020-10-13 | Nicira, Inc. | Static route configuration for logical router |
US10931560B2 (en) | 2018-11-23 | 2021-02-23 | Vmware, Inc. | Using route type to determine routing protocol behavior |
US10938788B2 (en) | 2018-12-12 | 2021-03-02 | Vmware, Inc. | Static routes for policy-based VPN |
US11252024B2 (en) | 2014-03-21 | 2022-02-15 | Nicira, Inc. | Multiple levels of logical routers |
US11283731B2 (en) | 2015-01-30 | 2022-03-22 | Nicira, Inc. | Logical router with multiple routing components |
US11425021B2 (en) | 2015-08-31 | 2022-08-23 | Nicira, Inc. | Authorization for advertised routes among logical routers |
US11539574B2 (en) | 2016-08-31 | 2022-12-27 | Nicira, Inc. | Edge node cluster network redundancy and fast convergence using an underlay anycast VTEP IP |
US20240095104A1 (en) * | 2022-09-15 | 2024-03-21 | Red Hat, Inc. | Asynchronous communication in cluster infrastructures |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6347177B2 (en) * | 2014-08-22 | 2018-06-27 | 富士通株式会社 | Transfer device, control device, and communication method |
US20160105329A1 (en) * | 2014-10-09 | 2016-04-14 | Splunk Inc. | Defining a service-monitoring dashboard using key performance indicators derived from machine data |
WO2018173246A1 (en) * | 2017-03-24 | 2018-09-27 | 株式会社日立製作所 | Storage system and storage control method |
CN109728932B (en) * | 2017-10-31 | 2021-09-28 | 中兴通讯股份有限公司 | SDN setting method, controller, switch and computer readable storage medium |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050021840A1 (en) * | 2003-07-11 | 2005-01-27 | Nokia Corporation | Method and an apparatus for enhancing messaging |
US20140348068A1 (en) * | 2011-09-20 | 2014-11-27 | Nokia Solutions And Networks Oy | Multiplexing Core Networks in RAN Sharing |
-
2012
- 2012-02-09 US US13/369,993 patent/US9130869B2/en active Active
-
2013
- 2013-02-04 WO PCT/IB2013/050946 patent/WO2013118049A1/en active Application Filing
- 2013-02-04 IN IN6960DEN2014 patent/IN2014DN06960A/en unknown
- 2013-02-04 EP EP13713971.3A patent/EP2813034B1/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050021840A1 (en) * | 2003-07-11 | 2005-01-27 | Nokia Corporation | Method and an apparatus for enhancing messaging |
US20140348068A1 (en) * | 2011-09-20 | 2014-11-27 | Nokia Solutions And Networks Oy | Multiplexing Core Networks in RAN Sharing |
Cited By (61)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130223277A1 (en) * | 2012-02-28 | 2013-08-29 | International Business Machines Corporation | Disjoint multi-pathing for a data center network |
US20160028611A1 (en) * | 2012-02-28 | 2016-01-28 | International Business Machines Corporation | Disjoint multi-pathing for a data center network |
US9185166B2 (en) * | 2012-02-28 | 2015-11-10 | International Business Machines Corporation | Disjoint multi-pathing for a data center network |
US9455899B2 (en) * | 2012-02-28 | 2016-09-27 | International Business Machines Corporation | Disjoint multi-pathing for a data center network |
US9178943B2 (en) * | 2012-02-28 | 2015-11-03 | International Business Machines Corporation | Disjoint multi-pathing for a data center network |
US20130223440A1 (en) * | 2012-02-28 | 2013-08-29 | International Business Machines Corporation | Disjoint multi-pathing for a data center network |
US9722922B2 (en) * | 2012-04-10 | 2017-08-01 | International Business Machines Corporation | Switch routing table utilizing software defined network (SDN) controller programmed route segregation and prioritization |
US20160028623A1 (en) * | 2012-04-10 | 2016-01-28 | International Business Machines Corporation | Switch routing table utilizing software defined network (sdn) controller programmed route segregation and prioritization |
US10228976B2 (en) * | 2012-05-01 | 2019-03-12 | Keysight Technologies Singapore (Holdings) Pte. Ltd. | Methods, systems, and computer readable media for balancing incoming connections across multiple cores |
US8908539B1 (en) * | 2012-05-25 | 2014-12-09 | Google Inc. | Systems and methods for testing network connections of a centrally-controlled network |
US9654375B1 (en) | 2012-05-25 | 2017-05-16 | Google Inc. | Systems and methods for testing network connections of a centrally-controlled network |
US9244868B2 (en) * | 2012-09-21 | 2016-01-26 | Avago Technologies General Ip (Singapore) Pte. Ltd. | Leased lock in active-active high availability DAS systems |
US20140089545A1 (en) * | 2012-09-21 | 2014-03-27 | Lsi Corporation | Leased lock in active-active high availability das systems |
US20140108632A1 (en) * | 2012-10-15 | 2014-04-17 | Cisco Technology, Inc. | System and method for efficient use of flow table space in a network environment |
US9548920B2 (en) * | 2012-10-15 | 2017-01-17 | Cisco Technology, Inc. | System and method for efficient use of flow table space in a network environment |
US20140169158A1 (en) * | 2012-12-17 | 2014-06-19 | Telefonaktiebolaget L M Ericsson (Publ) | Extending the reach and effectiveness of header compression in access networks using sdn |
US9246847B2 (en) * | 2012-12-17 | 2016-01-26 | Telefonaktiebolaget L M Ericsson (Publ) | Extending the reach and effectiveness of header compression in access networks using SDN |
US20150350156A1 (en) * | 2012-12-26 | 2015-12-03 | Zte Corporation | NAT implementation system, method, and Openflow switch |
US20140369259A1 (en) * | 2013-06-14 | 2014-12-18 | Samsung Electronics Co., Ltd. | Apparatus and method for transmitting/receiving data in mobile content network |
US9609684B2 (en) * | 2013-06-14 | 2017-03-28 | Samsung Electronics Co., Ltd. | Apparatus and method for transmitting/receiving data in mobile content network |
US9787586B2 (en) * | 2013-10-24 | 2017-10-10 | University Of Houston System | Location-based network routing |
US20150117458A1 (en) * | 2013-10-24 | 2015-04-30 | University Of Houston System | Location-based network routing |
JP2015104070A (en) * | 2013-11-27 | 2015-06-04 | エヌ・ティ・ティ・コムウェア株式会社 | Network management system, management device, controller, management method, control method, and program |
CN104717153A (en) * | 2013-12-12 | 2015-06-17 | 国际商业机器公司 | Resource over-subscription |
US9350678B2 (en) | 2014-01-16 | 2016-05-24 | International Business Machines Corporation | Controller based network resource management |
US9350677B2 (en) | 2014-01-16 | 2016-05-24 | International Business Machines Corporation | Controller based network resource management |
US11252024B2 (en) | 2014-03-21 | 2022-02-15 | Nicira, Inc. | Multiple levels of logical routers |
US10164894B2 (en) * | 2014-05-05 | 2018-12-25 | Nicira, Inc. | Buffered subscriber tables for maintaining a consistent network state |
US9602422B2 (en) | 2014-05-05 | 2017-03-21 | Nicira, Inc. | Implementing fixed points in network state updates using generation numbers |
US10091120B2 (en) | 2014-05-05 | 2018-10-02 | Nicira, Inc. | Secondary input queues for maintaining a consistent network state |
US20150319062A1 (en) * | 2014-05-05 | 2015-11-05 | Nicira, Inc. | Buffered subscriber tables for maintaining a consistent network state |
US20180081919A1 (en) * | 2014-09-02 | 2018-03-22 | Ab Initio Technology Llc | Compiling graph-based program specifications |
US10885003B2 (en) * | 2014-09-02 | 2021-01-05 | Ab Initio Technology Llc | Compiling graph-based program specifications |
US11301445B2 (en) | 2014-09-02 | 2022-04-12 | Ab Initio Technology Llc | Compiling graph-based program specifications |
US11283731B2 (en) | 2015-01-30 | 2022-03-22 | Nicira, Inc. | Logical router with multiple routing components |
US11343204B2 (en) | 2015-01-30 | 2022-05-24 | Nicira, Inc. | Datapath for multiple tenants |
US11706159B2 (en) | 2015-01-30 | 2023-07-18 | Nicira, Inc. | Datapath for multiple tenants |
US11799800B2 (en) | 2015-01-30 | 2023-10-24 | Nicira, Inc. | Logical router with multiple routing components |
CN105991606A (en) * | 2015-02-27 | 2016-10-05 | 中兴通讯股份有限公司 | OpenFlow message processing method and network element |
US9930010B2 (en) | 2015-04-06 | 2018-03-27 | Nicira, Inc. | Security agent for distributed network security system |
US20190075079A1 (en) * | 2015-04-06 | 2019-03-07 | Nicira, Inc. | Security cluster for performing security check |
US10142287B2 (en) * | 2015-04-06 | 2018-11-27 | Nicira, Inc. | Distributed network security controller cluster for performing security operations |
US11570147B2 (en) * | 2015-04-06 | 2023-01-31 | Nicira, Inc. | Security cluster for performing security check |
US20160294776A1 (en) * | 2015-04-06 | 2016-10-06 | Nicira, Inc. | Security cluster for performing security check |
US20200236054A1 (en) * | 2015-06-30 | 2020-07-23 | Huawei Technologies Co., Ltd. | Routing method, related device, and system |
US11509584B2 (en) * | 2015-06-30 | 2022-11-22 | Huawei Technologies Co., Ltd. | Routing method, related device, and system |
US11533256B2 (en) | 2015-08-11 | 2022-12-20 | Nicira, Inc. | Static route configuration for logical router |
US10805212B2 (en) | 2015-08-11 | 2020-10-13 | Nicira, Inc. | Static route configuration for logical router |
US11425021B2 (en) | 2015-08-31 | 2022-08-23 | Nicira, Inc. | Authorization for advertised routes among logical routers |
US10728094B2 (en) | 2015-10-28 | 2020-07-28 | Huawei Technologies Co., Ltd. | Control traffic in software defined networks |
WO2017071743A1 (en) * | 2015-10-28 | 2017-05-04 | Huawei Technologies Co., Ltd. | Control traffic in software defined networks |
US10795716B2 (en) | 2015-10-31 | 2020-10-06 | Nicira, Inc. | Static route types for logical routers |
US11593145B2 (en) | 2015-10-31 | 2023-02-28 | Nicira, Inc. | Static route types for logical routers |
US11418445B2 (en) | 2016-06-29 | 2022-08-16 | Nicira, Inc. | Installation of routing tables for logical router in route server mode |
US10749801B2 (en) * | 2016-06-29 | 2020-08-18 | Nicira, Inc. | Installation of routing tables for logical router in route server mode |
US11539574B2 (en) | 2016-08-31 | 2022-12-27 | Nicira, Inc. | Edge node cluster network redundancy and fast convergence using an underlay anycast VTEP IP |
CN106357450A (en) * | 2016-09-29 | 2017-01-25 | 全球能源互联网研究院 | Method for mapping asynchronous information of OpenFlow protocol |
US10931560B2 (en) | 2018-11-23 | 2021-02-23 | Vmware, Inc. | Using route type to determine routing protocol behavior |
US10797998B2 (en) | 2018-12-05 | 2020-10-06 | Vmware, Inc. | Route server for distributed routers using hierarchical routing protocol |
US10938788B2 (en) | 2018-12-12 | 2021-03-02 | Vmware, Inc. | Static routes for policy-based VPN |
US20240095104A1 (en) * | 2022-09-15 | 2024-03-21 | Red Hat, Inc. | Asynchronous communication in cluster infrastructures |
Also Published As
Publication number | Publication date |
---|---|
EP2813034B1 (en) | 2016-05-04 |
IN2014DN06960A (en) | 2015-04-10 |
US9130869B2 (en) | 2015-09-08 |
EP2813034A1 (en) | 2014-12-17 |
WO2013118049A1 (en) | 2013-08-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9130869B2 (en) | Methods of redirecting network forwarding elements and related forwarding elements and controllers | |
US11522790B2 (en) | Multipath data transmission processing method and network device | |
US10616379B2 (en) | Seamless mobility and session continuity with TCP mobility option | |
EP2823605B1 (en) | Methods of operating forwarding elements including shadow tables and related forwarding elements | |
US9742693B2 (en) | Dynamic service insertion in a fabric switch | |
US9112817B2 (en) | Efficient TRILL forwarding | |
US9385886B2 (en) | Method for processing a packet and related device | |
EP2985959B1 (en) | Progressive mac address learning | |
US9548873B2 (en) | Virtual extensible LAN tunnel keepalives | |
EP2940952A1 (en) | Method and system for facilitating switch virtualization in a network of interconnected switches | |
US10581758B2 (en) | Distributed hot standby links for vLAG | |
WO2012093429A1 (en) | Communication control system, control server, forwarding node, communication control method, and communication control program | |
CN108390821B (en) | Method and system for realizing dual activities of openflow switch | |
US20160241467A1 (en) | Method and system to optimize packet exchange between the control and data plane in a software defined network | |
US20210211351A1 (en) | Stacking-port configuration using zero-touch provisioning | |
US8953605B1 (en) | Methods and apparatus for a handshake protocol in a LAG-based multipath switch fabric for multipath OAM | |
WO2013176262A1 (en) | Packet transfer system, control device, packet transfer method and program | |
US8675669B2 (en) | Policy homomorphic network extension | |
JP6544242B2 (en) | PACKET TRANSFER SYSTEM, CONTROL DEVICE, RELAY DEVICE CONTROL METHOD, AND PROGRAM | |
WO2019196853A1 (en) | Tcp acceleration method and apparatus | |
US9935874B2 (en) | Fibre channel gateway system | |
US20200336434A1 (en) | Using bfd packets in a virtualized device | |
KR100729166B1 (en) | Switch port trunking control method in data communication system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: TELEFONAKTIEBOLAGET LM ERICSSON (PUBL), SWEDEN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MANGHIRMALANI, RAVI;SUBRAHMANIAM, RAMESH;REEL/FRAME:027900/0568 Effective date: 20120208 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
CC | Certificate of correction | ||
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |