GTMEssentials_V1.5.2.ppt

GTM Essentials,Prepared and presented by: LinJing V1.5.2 ,Updates,Updates,说明,该PPT适合对GTM/LTM已经有一定基础同学 如无GTM知识，建议首先阅读metoo的GTM安装指南-V2.0bible啊，下面连接是v1.0 请到 错别字请自行纠正 本文不是bible，F5软件日新月异，可能部分内容需基于指定版本 问题请在帖子下指出并讨论，感谢,GTM关键点,GTM中的对象关系 Listener决策策略 证书交换机制 Add new GTM to sync group八步法 iquery结构关系 gtmd、big3d的选举 部分重要算法 Monitor及path metric collection 私有地址server情形下的NAT处理 Named.conf,zone,wideip.conf,persistence的同步 ZoneRunner ;,ZoneRunner & BIND,由于使用动态更新方式控制zone，所以不要轻易尝试手工编辑zone文件 Wideip,zonerunner界面的配置修改，并不会立即同步到对应的zone文件中 这些变化临时存在于/var/named/config/namedb下对应zone的jnl文件中 BIND 的named.conf文件存放于/var/named/config目录下，zonerunner界面也可以编辑 F5的BIND默认不提供递归功能,ZoneRunner & BIND,See SOL7176:F5 Networks support for ZoneRunner, BIND, and the named daemon,GTM配置十步法,Define VLANs Define Self IPs Create default route Define NTP servers Define GTM listeners Create data centers Create Server objects big3d_install or bigip_add for BIG-IP servers Create GTM pool Objects Create WideIPs,日志检查及排错,Gtm日志保存在/var/log/gtm中 当需深入排错时，可以打开gtmd的一些高级日志 GTM.QueryLogging = enable /default disable GTM.DebugProbeLogging=enable /default disable Log.GTM.Level = debug /default notice Log.Big3d.Level = debug /defualt notic,日志检查及排错,rootb1500-930-1b:Active config # tail -f /var/log/gtm |grep 172.24.9.15 Jul 22 13:55:20 b1500-930-1b gtmd1036: 011ae039:7: Check probing of IP:Port 172.24.9.15:80 in DC dc3 Jul 22 13:55:20 b1500-930-1b gtmd1036: 011ae03a:7: Will not probe 172.24.9.15:80 in DC dc3 because will be done by other GTM (:172.24.9.12) Jul 22 13:55:31 b1500-930-1b gtmd1036: 011ae03d:5: Probe from 172.24.9.13: buffer = tcp :ffff:172.24.9.15 80 :ffff:172.24.9.15 80 6 4 2 state: timeout Jul 22 13:55:31 b1500-930-1b gtmd1036: 011ae0f2:1: Monitor instance tcp 172.24.9.15:80 UP - DOWN from 172.24.9.13 (state: timeout) Jul 22 13:55:31 b1500-930-1b gtmd1036: 011a6006:1: SNMP_TRAP: VS 172.24.9.15:80 (Server dc3_ext- host_training_server) state change green - red (VS dc3_ext- host_training_server: Monitor tcp from 172.24.9.13 : state: timeout) Jul 22 13:55:31 b1500-930-1b gtmd1036: 011a5004:1: SNMP_TRAP: Server dc3_ext- host_training_server (ip=172.24.9.15) state change green - red (Server dc3_ext- host_training_server: No enabled VS available) rootb1500-930-1b:Active ucs # tail -f /var/log/gtm |grep Wide Jul 22 13:55:31 b1500-930-1b gtmd1036: 011a3004:1: SNMP_TRAP: Wide IP state change green - red (Wide IP : No enabled pools available),日志检查及排错,rootb1500-925-1a:Active ucs # tail -f /var/log/gtm |grep 172.24.9.15 Jul 22 13:55:26 b1500-925-1a gtmd987: 011ae039:7: Check probing of IP:Port 172.24.9.15:80 in DC dc3 Jul 22 13:55:26 b1500-925-1a gtmd987: 011ae03b:7: Will probe 172.24.9.15:80 in DC dc3 Jul 22 13:55:26 b1500-925-1a gtmd987: 011ae03d:5: Probe to 172.24.9.13: buffer = tcp :ffff:172.24.9.15 80 :ffff:172.24.9.15 80 120 1 5 1 1 0 0 4 2 SEND= 1 RECV_I= 3 Jul 22 13:55:31 b1500-925-1a gtmd987: 011ae03d:5: Probe from 172.24.9.13: buffer = tcp :ffff:172.24.9.15 80 :ffff:172.24.9.15 80 6 4 2 state: timeout Jul 22 13:55:31 b1500-925-1a gtmd987: 011ae0f2:1: Monitor instance tcp 172.24.9.15:80 UP - DOWN from 172.24.9.13 (state: timeout) Jul 22 13:55:31 b1500-925-1a gtmd987: 011a6006:1: SNMP_TRAP: VS 172.24.9.15:80 (Server dc3_ext- host_training_server) state change green - red (VS dc3_ext- host_training_server: Monitor tcp from 172.24.9.13 : state: timeout) Jul 22 13:55:31 b1500-925-1a gtmd987: 011a5004:1: SNMP_TRAP: Server dc3_ext- host_training_server (ip=172.24.9.15) state change green - red (Server dc3_ext- host_training_server: No enabled VS available),日志检查及排错,Mar 17 05:55:48 gtm3 gtmd1260: 011ae03c:7: getconfig_put: Could not find my own box, will not continue with auto discovery. 把LTM加入到GTM，iquery都建立了，却没有vs被发现，如果有此日志说明gtm自己没有被加入到server中,日志检查及排错,Iqdump peer_ipaddress sync_group_name 不跟sync_group_name只能看到基本的心跳和server统计信息，无法看到vs信息 Dig或nslookup得到类似AUTHORITY: 2的flag，意味着结果是由BIND给出。,日志检查及排错,某个object由哪个big3d负责探测是可能不断变化的，这在某种情形下或许会导致server状态的flapping 在1500平台上，Big3d，gtmd竞争20%的CPU资源，因此大量的path探测可能会导致性能问题，可以关闭GTM上的big3d对path的探测功能，在加gtm到server的时候有iquery option，关闭path探测即可,日志检查及排错,SOL8187: Troubleshooting BIG-IP LTM and GTM device certificates SOL8195:Overview of the BIG-IP GTM big3d_install, bigip_add, and gtm_add utilities SOL4039:Overview of iQuery communication between BIG-IP and 3-DNS version 4.x and BIG-IP LTM and GTM version 9.x SOL5965:The BIG-IP GTM is unable to monitor BIG-IP version 4.x,日志检查及排错,wideip层面选择GA，GA所要选择的pool变红，但是该pool里的某个算法可以实现解析出一个IP地址，测试GTM依然会选择该POOL。 例如pool里的算法是fall back ip/return to dns. 因此如果要在WIDEIP层面做GA，应该不容许 pool层面总是可以得到结果 的情况出现。 Pool中算法选择none是控制算法跳过或跳跃到下一个可用pool的好方法。,Topology, ,Topology都miss的情况下 (注意备注),如果wideip层面的top算法都failed，则行为如下（V9.3.1 wideip下有2个pool）： 1.选择第一个可用的pool 2.按照被选择的pool里所设定的算法来作为pool间的算法，例如pool中是的第一算法是RR，则在POOL间轮询。 3。如果被选择pool的第一算法failed，则使用第二算法，因此类推。 4。如果被选择pool的算法所决定的最终算法是return to dns 则意味着这个pool不可用，此时return to到wideip上，而不是真正的BIND，即又会回到wideip层面选取下一个可用pool并按照该pool设定的算法执行，如果这个pool的算法最终又是return to dns，就会真正的return to BIND.,与排错相关的更多资源,GTM配置关键点及排错总结.pptx 这篇文档在给channel 的资料U盘上有 超超的GTM 2 days training - v2.pptx GTM部署模型方案讨论 ,关于开GTM CASE,NSE虽然通过wideip.conf可以大体复现一个网络结构，但是这往往不全面，提供全面清晰的GTM 网络结构图往往非常有助于解决问题 L3的服务，记得一定先自我仔细排查并搜集有效的信息及证据，准确描述问题现象及时间是关键 提供gtmd的debug及gtmd的probe debug日志信息往往更有效 Askf5,devcentral,拥有很多优秀的资源，常规问题或许别人已经见过 出现问题前做过什么，往往会有惊奇发现 相互理解更能快速解决问题,CASE-cert troubleshooting-clue,Iquery通信正常时候，gtmd,big3d之间的SSL过程： 以GTM和LTM为例，gtm要用其gtmd主动发起一个SSL到LTM上的big3d的协商，此过程中，GTM作为ssl的client端，ltm作为ssl的server端。这是一个双向SSL认证过程： GTM(gtmd)ssl client-LTM(big3d)ssl server 那么： 1.Gtmd发client hello 2.big3d响应server hello 并发送服务端cert （httpd证书），cert request（携带可接受的CA-从client.crt中读取）给gtmd 3.Gtmd得到big3d发来的证书后，用其server.crt来校验big3d发来的证书。并看big3d发来的可接受CA中是否有自己device cert所对应的CA。然后gtmd发送客户端cert（httpd证书）给big3d，并发送cert verify申请 4.big3d端检验gtmd发来的客户端证书并做hash校验 如果一切OK，则通过,正常时模拟iquery 连接,rootB3600-R18-S41:Active config # openssl s_client -tls1 -showcerts -connect 61.61.61.200:4353 -cert /config/httpd/conf/ssl.crt/server.crt -key /config/httpd/conf/ssl.key/server.key -verify /config/gtm/server.crt verify depth is 0 CONNECTED(00000003) depth=0 /C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20-server端发出的证书信息 verify error:num=18:self signed certificate -可以忽略 verify return:1 depth=0 /C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 verify return:1 - Certificate chain-server端证书链信息 0 s:/C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 i:/C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 -BEGIN CERTIFICATE- -下面的证书传给client MIICTDCCAbWgAwIBAgIBADANBgkqhkiG9w0BAQUFADBsMQswCQYDVQQGEwJDTjEQ MA4GA1UECBMHYmVpamluZzEQMA4GA1UEBxMHYmVpamluZzEOMAwGA1UEChMFZjVs dG0 xETAPBgNVBAsTCDIwMTAwNzAxMRYwFAYDVQQDEw1CMzQwMC1SMTctUzIwMB4X DTEwMDcwMTA1MjA1N1oXDTIwMDYyODA1MjA1N1owbDELMAkGA1UEBhMCQ04xEDAO BgNVBAgTB2JlaWppbmcxEDAOBgNVBAcTB2JlaWppbmcxDjAMBgNVBAoTBWY1bHRt MREwDwYDVQQLEwgyMDEwMDcwMTEWMBQGA1UEAxMNQjM0MDAtUjE3LVMyMDCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxy1oFb8B1wmBRUwJGwY1kCVlAlpqUMnH slOpA2aWNguADkI5P5cL2KSnoQvdt/6pX1WjJDhipA4yEJbCkYpmO+Lj28rzVKs+ 5FEa+IRDYsA/B6jUs5rwlzB07TWx8SZY5kLNX3IOzegOBnCw5Sllrm2rEdc3GCG2 ciQGC14wlCkCAwEAATANBgkqhkiG9w0BAQUFAAOBgQBq0yhPtxNdAfZlNgRw06cD X5B1lmZS/C+N12B8a5R9fpqaZ4VX5jbFZT/rxs8S/G9G6ch+b+RQHEW9L0D8yjc1 5l3Mii4eWH7mbjJBaJ7rDSRvQomYOIqIs+FZrXqmLdn75YNIc+wdrZcNiC6ud1Le cue+/sO7PKCPa+iZVSbjgQ= -END CERTIFICATE-,正常时模拟iquery 连接Cont.,- Server certificate -服务端证书信息及其issuer subject=/C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 issuer=/C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 - Acceptable client certificate CA names -服务端发出的cert request，表明其可接受这些CA下的证书 /C=-/ST=WA/L=Seattle/O=MyCompany/OU=1277774828/CN=localhost.localdomain/emailAddress=rootlocalhost.localdomain /C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 - SSL handshake has read 1040 bytes and written 1414 bytes - New, TLSv1/SSLv3, Cipher is AES256-SHA Server public key is 1024 bit Compression: zlib compression Expansion: zlib compression SSL-Session: Protocol : TLSv1 Cipher : AES256-SHA Session-ID: 63010946C033659C1F0D739320E9B514A2247F828B600D4DAC35D003308BA3B9 Session-ID-ctx: Master-Key: 930D767A9579440DE37D5CD45D989E9FD2BA646896669A33A068DD7531BC3CE85DCE77E82C45BEF70625409EBA43D702 Key-Arg : None Krb5 Principal: None Compression: 1 (zlib compression) Start Time: 1278039408 Timeout : 7200 (sec) Verify return code: 18 (self signed certificate) - x2?O?M.?K-*,a.?0?Qh?gg?glllS?. -握手通过后传出的压缩报文,CASE1：cert troubleshooting -clue,环境：一个GTM，一个LTM，bigip_add同步证书后，应该在GTM的server.crt里有GTM+LTM的证书，在LTM的client.crt里有GTM+LTM的证书 确认证书同步没错，iquery正确，LTM也绿色 在GTM的trusted server certificate中删掉LTM的证书，然后重启LTM的big3d，使iquery重连。 在GTM上openssl发起连接测试，得到下页结果,在gtm-server.crt中删除ltmcert测试,verify depth is 0 CONNECTED(00000003） depth=0 /C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 verify error:num=18:self signed certificate- verify return:1 depth=0 /C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 verify return:1 - Certificate chain 0 s:/C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 i:/C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 -BEGIN CERTIFICATE- MIICTDCCAbWgAwIBAgIBADANBgkqhkiG9w0BAQUFADBsMQswCQYDVQQGEwJDTjEQ MA4GA1UECBMHYmVpamluZzEQMA4GA1UEBxMHYmVpamluZzEOMAwGA1UEChMFZjVs dG0 xETAPBgNVBAsTCDIwMTAwNzAxMRYwFAYDVQQDEw1CMzQwMC1SMTctUzIwMB4X DTEwMDcwMTA1MjA1N1oXDTIwMDYyODA1MjA1N1owbDELMAkGA1UEBhMCQ04xEDAO BgNVBAgTB2JlaWppbmcxEDAOBgNVBAcTB2JlaWppbmcxDjAMBgNVBAoTBWY1bHRt MREwDwYDVQQLEwgyMDEwMDcwMTEWMBQGA1UEAxMNQjM0MDAtUjE3LVMyMDCBnzAN BgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxy1oFb8B1wmBRUwJGwY1kCVlAlpqUMnH slOpA2aWNguADkI5P5cL2KSnoQvdt/6pX1WjJDhipA4yEJbCkYpmO+Lj28rzVKs+ 5FEa+IRDYsA/B6jUs5rwlzB07TWx8SZY5kLNX3IOzegOBnCw5Sllrm2rEdc3GCG2 ciQGC14wlCkCAwEAATANBgkqhkiG9w0BAQUFAAOBgQBq0yhPtxNdAfZlNgRw06cD X5B1lmZS/C+N12B8a5R9fpqaZ4VX5jbFZT/rxs8S/G9G6ch+b+RQHEW9L0D8yjc1 5l3Mii4eWH7mbjJBaJ7rDSRvQomYOIqIs+FZrXqmLdn75YNIc+wdrZcNiC6ud1Le cue+/sO7PKCPa+iZVSbjgQ= -END CERTIFICATE-,接上页,- Server certificate subject=/C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 issuer=/C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 - Acceptable client certificate CA names /C=-/ST=WA/L=Seattle/O=MyCompany/OU=1277774828/CN=localhost.localdomain/emailAddress=rootlocalhost.localdomain /C=CN/ST=beijing/L=beijing/O=f5ltm/OU=20100701/CN=B3400-R17-S20 - SSL handshake has read 1040 bytes and written 1414 bytes - New, TLSv1/SSLv3, Cipher is AES256-SHA Server public key is 1024 bit Compression: zlib compression Expansion: zlib compression SSL-Session: Protocol : TLSv1 Cipher : AES256-SHA Session-ID: 00DC03FFA492BE4C00A2D57043E5AFA6801FEE4E897483A60E464434039E4EE2 Session-ID-ctx: Master-Key: 8D0FF68C39573FE57B7018B9117F7FD04BEAF42033A5C3F2BBA0592BE022ED890E5AEA66F3473959D22B34DF12B42F0A Key-Arg : None Krb5 Principal: None Compression: 1 (zlib compression) Start Time: 1278034050 Timeout : 7200 (sec) Verify return code: 18 (self signed certificate) - x2?O?M.?K-*,a.?0?Qh?gg?glllS?,看iqdump输出的技巧,rootB3600-R18-S41:Active config # iqdump 61.61.61.200 3466:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:s3_clnt.c:897: -第一行给出最终的SSL错误错在哪！这很重要，例如上面信息说明，client提交证书后请求cert verify，但最终server认为verify出错 （客户端lgtmd上关于ltm的证书被删除了-顺延上面的ppt） - New, (NONE), Cipher is (NONE) SSL-Session: Protocol : TLSv1 Cipher : 0000 Session-ID: 1B7E30B22EF72147A34177B7DF5740B4123A0042EFDD94EA84C3C19FA9EB0AF7 Session-ID-ctx: Master-Key: Key-Arg : None Krb5 Principal: None Start Time: 1278042375 Timeout : 7200 (sec) Verify return code: 0 (ok) - 上面这一块给出SSL的协商会话信息，如果这里的return code没错，往往意味着SSL握手过程没错，但不代表就真的SSL通信正常,看iqdump输出的技巧-cont.,见备注 基于P62所述环境的输出,思想:bgi3d作为SSL server端,gtmd作为ssl client端，从SSL握手验证角度考虑证书验证关系： GTM自己验证自己不用考虑根证问题,Some case,C713247,v10.1（ltm combo gtm）,不要将ltm中不存在的vs （即generic host）定义到gtm的server下，会导致一些其他的vs flapping. C681773,v10.1某种情形下同步触发后会导致wideip文件出现乱字符，Enhotfix C708249 v10.1作为委派的NS record建立后，GUI不能点击进去查看该记录，显示“Resolver returned no such record.” 已知问题，以后版本修复 SOL12062 不要在gtm的vs层面设置snmp monitor,SG gtm training 2011、7.7,tcpdump -ni 0.0:nnn s0 port 53 tcpdump -ni tmm0 s0 port 53 Tmm0可以抓取从tmm发给bind的包,