[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Sks-devel] [PATCH] IPv6 support
|
From: |
Phil Pennock |
|
Subject: |
[Sks-devel] [PATCH] IPv6 support |
|
Date: |
Wed, 4 Mar 2009 23:24:46 -0800 |
On 2009-03-02 at 02:52 -0800, Phil Pennock wrote:
> Can those with more experience in OCaml please take a look over this and
> tell me if I'm heading down the right path?
>
> This patch theoretically adds support for IPv6 in recon; it adds the
> settings hooks for hkp too, but doesn't add anything that uses that yet.
> It should both listen on IPv6 and connect outbound too.
>
> It compiles. It runs. I see both listening sockets with lsof. It
> doesn't do anything soon enough for me to see tonight how well I've
> done, as I need to go get some sleep. I'm not sufficiently comfortable
> to leave sks-peer.spodhuis.org running this code overnight, so I've
> reverted to 1.1.0.
>
> It's my first OCaml hacking and my style sucks and I spend more time
> debugging to get it to compile than I care to think about.
>
> A critical eye and constructive comments appreciated.
>
> Oh, and I've no idea how recent support for IPv6 in OCaml is and have
> made no effort to gracefully degrade to either old OCaml environments or
> systems without IPv6 support. First, I want to get it working for
> anyone, *then* I can worry about making it conditional.
I've now extended the code to implement IPv6 for hkp, not just recon,
and to make outbound connections fine.
People have a choice:
* Kim's simpler patch, which relies upon the OS supporting IPv6-mapped
IPv4 addresses; the *BSD systems disable this by default, but you can
change this system-wide by setting sysctl net.inet6.ip6.v6only to 0.
I believe that OCaml lacks the necessary support in its setsockopt
shim to set this option, so it can't be worked around
* My larger patch, which opens separate sockets for each stack and
which adds recon6_address and hkp6_address options to sksconf if you
want to use a binding address other than :: (the IPv6 equivalent to
0.0.0.0).
In particular, if anyone has an IPv4-only system, I would like to know
what problems are caused by my patch, but I'll fully understand if
people don't want to try that out. :)
I'm now running my patch live, eating my own dogfood. I've used gpg to
get a key from the HKP port over IPv6, but not yet seen recon, in part
because my DNS didn't include an AAAA record. I've updated my DNS and
we're now past the TTL expiry.
$ gpg --keyserver 'hkp://[2001:980:fff:31::10]' --recv-key $keyid
I am not providing a Mercurial repository, as I'm not sufficiently
familiar with the system to assess the security risks of doing so and
don't want to leave up a daemon which I'd access so rarely. If I used
Mercurial on a regular basis, that would be a different matter.
So, attached, please find what I currently believe to be a complete IPv6
support patch. Unless you define "complete" to mean "can disable it".
I would have sent this mail 2 hours ago but I realised that the
membership code just taking the first address found was a bad plan, as
then there would be a 50% chance of a connection from a mutually IPv6
host failing -- if they connected over IPv4 and you only record the IPv6
address, that would be rejected. So I've just broken my brain on
O'Caml's type system while making sure that the membership handling will
track _all_ the IP addresses of each peer.
Note that one side-effect is that in theory, with this, you could peer
with a cluster of hosts where one name has multiple IPs. But you
wouldn't want to do that normally, especially since the "that's myself"
test will drop any item where any of the IP addresses are any of the
local recon addresses (either the one configured address or any of the
addresses for hostname). Oh, and that would possibly not be a good plan
from the point of view of the synchronisation protocol itself.
With this latest fix, I'm not rejecting so many recon connects now. ;)
If you try my patch out, I'd appreciate feedback. If you're peering
with sks-peer.spodhuis.org then you might want to keep an eye on
recon.log and see if you get an IPv6 peering. :) But please, keep a
safe copy of your normal binary to fall back to.
Thanks,
-Phil
diff -r e30dc5376cbb common.ml
--- a/common.ml Fri May 23 21:16:40 2008 -0400
+++ b/common.ml Wed Mar 04 23:06:51 2009 -0800
@@ -199,8 +199,10 @@
let recon_port = !Settings.recon_port
let recon_address = !Settings.recon_address
+let recon6_address = !Settings.recon6_address
let http_port = !Settings.hkp_port
let http_address = !Settings.hkp_address
+let http6_address = !Settings.hkp6_address
let db_command_name = Filename.concat !Settings.basedir "db_com_sock"
let recon_command_name = Filename.concat !Settings.basedir "recon_com_sock"
@@ -212,7 +214,15 @@
| Unix.ADDR_INET (inet_addr,port) -> Unix.ADDR_INET (inet_addr,port + 1)
-let get_client_recon_addr () =
- Unix.ADDR_INET (Unix.inet_addr_of_string recon_address,0)
+let get_client_recon_addr remote_addr = match Unix.domain_of_sockaddr
remote_addr with
+ Unix.PF_UNIX -> failwith "Can't connect to remote local-domain sockets"
+ | Unix.PF_INET -> Unix.ADDR_INET (Unix.inet_addr_of_string recon_address,0)
+ | Unix.PF_INET6 -> Unix.ADDR_INET (Unix.inet_addr_of_string recon6_address,0)
let get_client_recon_addr =
- Utils.unit_memoize get_client_recon_addr
+ Utils.memoize get_client_recon_addr
+
+let sockaddr_to_string sockaddr = match sockaddr with
+ Unix.ADDR_UNIX s -> sprintf "<ADDR_UNIX %s>" s
+ | Unix.ADDR_INET (addr,p) -> sprintf "<ADDR_INET [%s]:%d>"
+ (Unix.string_of_inet_addr addr) p
+
diff -r e30dc5376cbb dbserver.ml
--- a/dbserver.ml Fri May 23 21:16:40 2008 -0400
+++ b/dbserver.ml Wed Mar 04 23:06:51 2009 -0800
@@ -57,8 +57,10 @@
failwith "Running sks_db without transactions is no longer supported."
- let addr = inet_addr_of_string http_address
- let websock = Eventloop.create_sock (ADDR_INET (addr,http_port))
+ let addr4 = inet_addr_of_string http_address
+ let addr6 = inet_addr_of_string http6_address
+ let websock4 = Eventloop.create_sock (ADDR_INET (addr4,http_port))
+ let websock6 = Eventloop.create_sock (ADDR_INET (addr6,http_port))
let () =
if Sys.file_exists db_command_name
then Unix.unlink db_command_name
@@ -652,7 +654,11 @@
)
(
- (websock, Eventloop.make_th ~name:"webserver"
+ (websock4, Eventloop.make_th ~name:"webserver (IPv4)"
+ ~timeout:!Settings.wserver_timeout
+ ~cb:(Wserver.accept_connection webhandler ~recover_timeout:1))
+ ::
+ (websock6, Eventloop.make_th ~name:"webserver (IPv6)"
~timeout:!Settings.wserver_timeout
~cb:(Wserver.accept_connection webhandler ~recover_timeout:1))
::
@@ -661,11 +667,16 @@
~cb:(eventify_handler command_handler))
::
(if !Settings.use_port_80 then
- let sock = Eventloop.create_sock (ADDR_INET (addr,80)) in
- (sock,Eventloop.make_th ~name:"webserver80"
+ let sock4 = Eventloop.create_sock (ADDR_INET (addr4,80)) in
+ let sock6 = Eventloop.create_sock (ADDR_INET (addr6,80)) in
+ (sock4,Eventloop.make_th ~name:"webserver80 (IPv4)"
~timeout:!Settings.wserver_timeout
- ~cb:(Wserver.accept_connection webhandler ~recover_timeout:1)
- )::[]
+ ~cb:(Wserver.accept_connection webhandler ~recover_timeout:1))
+ ::
+ (sock6,Eventloop.make_th ~name:"webserver80 (IPv6)"
+ ~timeout:!Settings.wserver_timeout
+ ~cb:(Wserver.accept_connection webhandler ~recover_timeout:1))
+ ::[]
else
[]
)
diff -r e30dc5376cbb eventloop.ml
--- a/eventloop.ml Fri May 23 21:16:40 2008 -0400
+++ b/eventloop.ml Wed Mar 04 23:06:51 2009 -0800
@@ -121,10 +121,7 @@
let create_sock addr =
try
- let domain =
- match addr with
- ADDR_UNIX _ -> PF_UNIX
- | ADDR_INET (_,_) -> PF_INET in
+ let domain = Unix.domain_of_sockaddr addr in
let sock =
socket ~domain ~kind:SOCK_STREAM ~protocol:0 in
setsockopt sock SO_REUSEADDR true;
@@ -132,8 +129,10 @@
listen sock ~max:20;
sock
with
- | Unix_error (_,"bind",_) ->
- failwith "Failure while binding socket. Probably another socket bound
to this address"
+ | Unix_error (error, "bind",_) ->
+ let msg = error_message error in
+ let addrstr = sockaddr_to_string addr in
+ failwith (sprintf "Failure binding %s: %s" addrstr msg)
| e -> raise e
let add_events heap evlist =
List.iter ~f:(fun (Event (time, callback)) ->
diff -r e30dc5376cbb membership.ml
--- a/membership.ml Fri May 23 21:16:40 2008 -0400
+++ b/membership.ml Wed Mar 04 23:06:51 2009 -0800
@@ -29,38 +29,112 @@
exception Bug of string
exception Lookup_failure of string
exception Malformed_entry of string
+exception Empty_line
let membership = ref ([| |],-1.)
let whitespace = Str.regexp "[ \t]+"
-let lookup_hostname string =
- try (Unix.gethostbyname string).Unix.h_addr_list.(0)
- with
+(* Although the OCaml standard Unix library does not include AF_UNSPEC, the
+ * underlying C implementation uses AF_UNSPEC unless overriden (ie, there's
+ * no way to specify a value which matches the default behaviour).
+ * So for IPv4/IPv6 functionality, use getaddrinfo without an AI_FAMILY
+ * parameter.
+ * This does mean that we need a separate function to filter though.
+ *)
+
+let inet_addr_of_sockaddr sock = match sock with
+ Unix.ADDR_UNIX _ -> raise (Bug "asking for inet addr of local domain
socket")
+ | Unix.ADDR_INET (addr, port) -> addr
+
+let inet_addr_of_addr_info ai = match ai.Unix.ai_addr with
+ Unix.ADDR_UNIX _ -> raise (Bug "converting getaddrinfo result which is
local domain?")
+ | Unix.ADDR_INET (addr, port) -> addr
+
+let lookup_hostname_as_ai_list ?(port = recon_port) string =
+ let portstr = string_of_int port in
+ try begin
+ let reslist = Unix.getaddrinfo string portstr [Unix.AI_SOCKTYPE
Unix.SOCK_STREAM] in
+ if List.length reslist == 0 then raise (Lookup_failure string)
+ else List.map (fun (a) -> a.Unix.ai_addr) reslist
+ end
+ with
| Invalid_argument _ | Not_found -> raise (Lookup_failure string)
-let local_recon_addr () =
- Unix.ADDR_INET (lookup_hostname !Settings.hostname, recon_port)
+let lookup_hostname_list_filtered ~family ?(port = recon_port) string =
+ let portstr = string_of_int port in
+ let filter = (fun ai -> if ai.Unix.ai_family == family then true else false)
in
+ try begin
+ let reslist = Unix.getaddrinfo string portstr [Unix.AI_SOCKTYPE
Unix.SOCK_STREAM] in
+ if List.length reslist == 0 then raise (Lookup_failure string)
+ else List.map inet_addr_of_addr_info (List.filter filter reslist)
+ end
+ with
+ | Invalid_argument _ | Not_found -> raise (Lookup_failure string)
-let local_recon_addr = Utils.unit_memoize local_recon_addr
+let my_ipv4_recon_addresses () =
+ let r = Unix.inet_addr_of_string !Settings.recon_address in
+ if r = Unix.inet_addr_any
+ then lookup_hostname_list_filtered ~family:Unix.PF_INET !Settings.hostname
+ else [r]
+let my_ipv4_recon_addresses = Utils.unit_memoize my_ipv4_recon_addresses
-let remove_self addresses =
- List.filter ~f:(fun (addr,str) -> addr <> local_recon_addr ()) addresses
+let my_ipv6_recon_addresses () =
+ let r = Unix.inet_addr_of_string !Settings.recon6_address in
+ if r = Unix.inet6_addr_any
+ then lookup_hostname_list_filtered ~family:Unix.PF_INET6 !Settings.hostname
+ else [r]
+let my_ipv6_recon_addresses = Utils.unit_memoize my_ipv6_recon_addresses
-let convert_address l =
+let is_local_recon_addr peer = match Unix.domain_of_sockaddr peer with
+ Unix.PF_UNIX -> raise (Bug "trying to find local address of remote local
domain socket")
+ | Unix.PF_INET -> List.exists (fun a -> (inet_addr_of_sockaddr peer) = a)
(my_ipv4_recon_addresses ())
+ | Unix.PF_INET6 -> List.exists (fun a -> (inet_addr_of_sockaddr peer) = a)
(my_ipv6_recon_addresses ())
+
+let is_local_recon_addr = Utils.memoize is_local_recon_addr
+
+(* We have to support storing all INET_ADDR items for a given peer, so that
+ * membership tests will accept connections from both of a peer's addresses,
+ * instead of accepting only one, vaguely randomly.
+ *
+ * So while previously we stored only one IP address for a peer, we now record
+ * all IP addresses, both v4 and v6, which also lets a peer be multi-addressed
+ * for resilient service. Bonus!
+ *
+ * The internal membership storage is only accessed within membership.ml, so
+ * we can change the storage without changing the API. Thus we just replace
+ * storing "an address" with "a list of addresses" and update the test
+ * interfaces too.
+ *)
+
+let convert_hostname_to_sockaddrs l =
try
- sscanf l "%s %d"
- (fun addr port -> Unix.ADDR_INET (lookup_hostname addr,port))
+ if String.length l == 0 then raise Empty_line
+ else
+ sscanf l "%s %d"
+ (fun addr port -> lookup_hostname_as_ai_list ~port:port addr)
with
Scanf.Scan_failure _ | End_of_file | Failure _ -> raise (Malformed_entry l)
+(* Note that we filter out any peers where any of our own IPs is one of the
+ * IPs in DNS for the peer, so if you're running a cluster of servers you'll
+ * internally need to use split hostnames for each instance in your membership.
+ *)
+
+let remove_self membership_items =
+ let check_list addr_list =
+ not (List.exists ~f:is_local_recon_addr addr_list)
+ in
+ List.filter ~f:(fun (addrlist, str) -> check_list addrlist) membership_items
+
let load_membership_file file =
let rec loop list =
try
let line = decomment (input_line file) in
- let addr = convert_address line in
- (addr,line) :: loop list
+ let sockaddr_list = convert_hostname_to_sockaddrs line in
+ (sockaddr_list,line) :: loop list
with
+ | Empty_line -> loop list
| End_of_file -> list
| Lookup_failure addr ->
perror "Lookup failure on address %s" addr;
@@ -93,15 +167,13 @@
)
~finally:(fun () -> close_in file)
-let sockaddr_to_string sockaddr = match sockaddr with
- Unix.ADDR_UNIX s -> sprintf "<ADDR_UNIX %s>" s
- | Unix.ADDR_INET (addr,p) -> sprintf "<ADDR_INET %s:%d>"
- (Unix.string_of_inet_addr addr) p
-
let membership_string () =
let (mshp,_) = !membership in
- let to_string (addr,str) =
- sprintf "%s(%s)" (sockaddr_to_string addr) str
+ let to_string (addr_list,str) =
+ String.concat " " (
+ "(" :: (
+ List.map (fun (a) -> (sockaddr_to_string a)) addr_list
+ ) @ str :: ")" :: [] )
in
let strings = List.map ~f:to_string (Array.to_list mshp) in
"Membership: " ^ String.concat ~sep:", " strings
@@ -147,19 +219,25 @@
in
Array.map ~f:fst mshp
-let same_inet_addr addr1 addr2 =
- match (addr1,addr2) with
- (Unix.ADDR_INET (ip1,_), Unix.ADDR_INET (ip2,_)) -> ip1 = ip2
- | _ -> false
+(* Used to test if a peer is in our membership file.
+ * When this was IPv4-only, it sufficed to test a single address against
+ * a single address. Now, we want to check for a non-empty intersection
+ * of two sets of addresses. We don't use real Set stuff because the
+ * total number of addresses is expected to be small, typically 1 or 2.
+ *)
+
+let ipmatch_ip_in_socklist ip1 sockaddr_list =
+ List.exists ~f:(fun (a2) -> ip1 = (inet_addr_of_sockaddr a2)) sockaddr_list
let test addr =
reload_if_changed ();
let (m,mtime) = !membership in
+ let ip = inet_addr_of_sockaddr addr in
let found = ref false in
let i = ref 0 in
while !i < Array.length m && not !found do
- if same_inet_addr addr (fst m.(!i)) then
+ if ipmatch_ip_in_socklist ip (fst m.(!i)) then
found := true;
incr i
done;
diff -r e30dc5376cbb reconCS.ml
--- a/reconCS.ml Fri May 23 21:16:40 2008 -0400
+++ b/reconCS.ml Wed Mar 04 23:06:51 2009 -0800
@@ -129,15 +129,17 @@
(** function to connect to remote host to initate reconciliation *)
-let connect tree ~filters ~partner ~self =
+let connect tree ~filters ~partner =
(* TODO: change the following to depend on the address type *)
+ let peer_addr = get_client_recon_addr partner in
+ let sock_domain = Unix.domain_of_sockaddr partner in
let s = Unix.socket
- ~domain:Unix.PF_INET
- ~kind:Unix.SOCK_STREAM
- ~protocol:0
+ ~domain:sock_domain
+ ~kind:Unix.SOCK_STREAM
+ ~protocol:0
in
let run () =
- Unix.bind s ~addr:(get_client_recon_addr ());
+ Unix.bind s ~addr:peer_addr;
Unix.connect s ~addr:partner;
let cin = Channel.sys_in_from_fd s
and cout = Channel.sys_out_from_fd s in
diff -r e30dc5376cbb reconComm.ml
--- a/reconComm.ml Fri May 23 21:16:40 2008 -0400
+++ b/reconComm.ml Wed Mar 04 23:06:51 2009 -0800
@@ -67,11 +67,11 @@
let get_keystrings_via_http addr hashes =
let s = Unix.socket
- ~domain:Unix.PF_INET
+ ~domain:(Unix.domain_of_sockaddr addr)
~kind:Unix.SOCK_STREAM
~protocol:0 in
protect ~f:(fun () ->
- Unix.bind s ~addr:(get_client_recon_addr ());
+ Unix.bind s ~addr:(get_client_recon_addr addr);
Unix.connect s ~addr;
let cin = Channel.sys_in_from_fd s
and cout = Channel.sys_out_from_fd s in
diff -r e30dc5376cbb reconserver.ml
--- a/reconserver.ml Fri May 23 21:16:40 2008 -0400
+++ b/reconserver.ml Wed Mar 04 23:06:51 2009 -0800
@@ -50,7 +50,9 @@
(******************************************************************)
let recon_addr = Unix.ADDR_INET (Unix.inet_addr_of_string
recon_address,recon_port)
+ let recon6_addr = Unix.ADDR_INET (Unix.inet_addr_of_string
recon6_address,recon_port)
let reconsock = Eventloop.create_sock recon_addr
+ let recon6sock = Eventloop.create_sock recon6_addr
let () =
if Sys.file_exists recon_command_name
@@ -78,9 +80,13 @@
if Array.length array = 0 then raise Not_found
else array.(Random.int (Array.length array))
+ let choose_addr list =
+ if List.length list = 0 then raise Not_found
+ else List.nth list (Random.int (List.length list))
+
let choose_partner () =
try
- choose (Membership.get ())
+ choose_addr (choose (Membership.get ()))
with
Not_found | Invalid_argument _ ->
failwith "No gossip partners available"
@@ -215,7 +221,6 @@
let filters = get_filters () in
let (results,http_addr) =
ReconCS.connect (get_ptree ()) ~filters ~partner
- ~self:Membership.local_recon_addr
in
let results = ZSet.elements results in
plerror 4 "Reconciliation complete";
@@ -360,7 +365,12 @@
~timeout:!Settings.command_timeout
);
(reconsock, Eventloop.make_th
- ~name:"reconciliation handler"
+ ~name:"reconciliation handler (IPv4)"
+ ~cb:recon_handler
+ ~timeout:!Settings.reconciliation_config_timeout
+ );
+ (recon6sock, Eventloop.make_th
+ ~name:"reconciliation handler (IPv6)"
~cb:recon_handler
~timeout:!Settings.reconciliation_config_timeout
);
diff -r e30dc5376cbb settings.ml
--- a/settings.ml Fri May 23 21:16:40 2008 -0400
+++ b/settings.ml Wed Mar 04 23:06:51 2009 -0800
@@ -56,12 +56,16 @@
seed := value
let recon_port = ref 11370
-let recon_address = ref "0.0.0.0"
+let recon_address = ref (Unix.string_of_inet_addr Unix.inet_addr_any)
let set_recon_address value = recon_address := value
+let recon6_address = ref (Unix.string_of_inet_addr Unix.inet6_addr_any)
+let set_recon6_address value = recon6_address := value
let hkp_port = ref 11371
-let hkp_address = ref "0.0.0.0"
+let hkp_address = ref (Unix.string_of_inet_addr Unix.inet_addr_any)
let set_hkp_address value = hkp_address := value
+let hkp6_address = ref (Unix.string_of_inet_addr Unix.inet6_addr_any)
+let set_hkp6_address value = hkp6_address := value
let use_port_80 = ref false
@@ -247,9 +251,11 @@
("-baseport",Arg.Int set_base_port, " Set base port number");
("-logfile",Arg.String (fun _ -> ()), " DEPRECATED. Now ignored.");
("-recon_port",Arg.Int set_recon_port, " Set recon port number");
- ("-recon_address",Arg.String set_recon_address, " Set recon binding
address");
+ ("-recon_address",Arg.String set_recon_address, " Set recon binding
address (IPv4)");
+ ("-recon6_address",Arg.String set_recon6_address, " Set recon binding
address (IPv6)");
("-hkp_port",Arg.Int set_hkp_port, " Set hkp port number");
- ("-hkp_address",Arg.String set_hkp_address, " Set hkp binding address");
+ ("-hkp_address",Arg.String set_hkp_address, " Set hkp binding address
(IPv4)");
+ ("-hkp6_address",Arg.String set_hkp6_address, " Set hkp binding address
(IPv6)");
("-use_port_80",Arg.Set use_port_80,
" Have the HKP interface listen on port 80, as well as the hkp_port");
("-basedir", Arg.Set_string basedir, " Base directory");
diff -r e30dc5376cbb sks.pod
--- a/sks.pod Fri May 23 21:16:40 2008 -0400
+++ b/sks.pod Wed Mar 04 23:06:51 2009 -0800
@@ -152,7 +152,11 @@
=item -recon_address
-Set recon binding address.
+Set recon binding IPv4 address. Also used for outbound connections.
+
+=item -recon6_address
+
+Set recon binding IPv6 address. Also used for outbound connections.
=item -hkp_port
@@ -160,7 +164,11 @@
=item -hkp_address
-Set hkp binding address.
+Set hkp binding IPv4 address.
+
+=item -hkp6_address
+
+Set hkp binding IPv6 address.
=item -use_port_80
pgpGDhW1ZihwA.pgp
Description: PGP signature
Re: [Sks-devel] [PATCH] IPv6 support, Kim Minh Kaplan, 2009/03/07