Author: Sohail Qamar

CVE-2021-44228 and CVE-2021-45046 workaround instructions for VMWare VCenter

It has been established that CVE-2021-44228 and CVE-2021-45046 affect vCenter Server 7.0.x, vCenter 6.8.x, and vCenter 6.5.x via the Apache Log4j open source component that it ships. Please read the VMware Security Advisory (VMSA) below before moving to the next step to learn more about this vulnerability and its potential impact on VMware products.

  • Before following the instructions in this article, VCHA must be deleted. It can be rearranged at a later point.
  • Steps must be done on both vCenter and PSC appliances in environments with external PSCs.
  • A File-Based Backup restore will put the environment back in danger. After restoring, use the vc_log4j_mitigator.py script to fix this.
  • The environment will become vulnerable once more if the vCenter Appliance is upgraded to an untreated version. Correct this issue after upgrading by running the vc_log4j_mitigator.py script.

The CVE-2021-44228 and CVE-2021-45046 vulnerability are resolved in:

  • vCenter Server 7.0 Update 3c, build 19234570
  • vCenter Server 6.7 Update 3q, build 19300125
  • vCenter Server 6.5 Update 3s, build 19261680

Before upgrading to a newer version of vCenter Server, you don’t need to revert the workaround procedures in this article.

Not to be used on any vCenter Server that has been upgraded to a fixed version, such as 7.0 U3c, the vc_log4j_mitigator.py script.

The workarounds suggested in this document are designed to be a temporary solution only.

WARNING: End-to-end mitigation for vc_log4j_mitigator.py for CVE-2021-45046 and CVE-2021-44228 has been added. As a result of this script, vmsa-2021-0028-kb87081 and remove_log4j_class.py no longer need to be executed separately. In this case, however, it is not required to perform the action if you have already done so.

Automated Workaround

For automated remediation of CVE-2021-44228 and CVE-2021-45046 copy and paste the following python script in  a file named vc_log4j_mitigator.py

  • Login to VCSA using an SSH client such as Putty.
  • Transfer the file vc_log4j_mitigator.py that you have earlier saved to /tmp folder on the VCenter server Appliance.
  • Make sure the enable the bash shell in Vcenter.
  • Execute the script vc_log4j_mitigator.py that you have earlier saved in /tmp folder as follows.

python vc_log4j_mitigator.py

All vCenter services will be stopped, all relevant files will be updated with the formatMsgNoLookups flag, the JndiLookup.class will be removed from all jar/war files on the appliance, and lastly all vCenter services will be started. As the script runs, the files that it edits will be reported.

  • Execute the script again using the dry run flag to ensure there are no more susceptible files.

python vc_log4j_mitigator.py -r

  • After the script run finishes, the list of vulnerable files must be zero.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
#!/usr/bin/env python
"""
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
===
Handles CVE-2021-44228 exploit for VMware vCenter Server for both Virtual
Appliance and Microsoft Windows environments.
This script can run in its default remediation mode or in dry run scan mode.
Remediation mode applies fixes to the vCenter configuration and library code.
Dry run scan mode is used to identify vulnerable files and validate the fixes
from a remediation run.
The steps in the remediation mode are:
1. Stop all services to unlock all configuration and library files that may
need remediation. This step uses "service-control --stop --all" command.
2. Scan Java library files (*.jar and *.war) for "JndiLookup.class" and remove it.
3. Based on the vCenter system version, deployment flavour and OS type scans the
configuration files for "-Dlog4j2.formatMsgNoLookups=true" configuration
option and adds it as needed.
4. Start all services using "service-control --start --all"
In the remediation mode any modified files are backed up. The backup location
needs to be set with "-b" option or will default to a temporary folder that must
be backed up immediately after running the script.
In dry run scan mode the script will only examine the files from steps 2 and 3
above and report any potentially vulnerable files. No service stop and start is
needed in dry run mode. The dry run mode is activated with "-r".
The script produces detailed log file under the VMWARE_LOG_DIR folder.
"""
import os
import sys
import shutil
from distutils.file_util import copy_file
import zipfile
import codecs
import tempfile
import subprocess
import logging
import argparse
import hashlib
import json
from datetime import datetime
from itertools import chain
import re
LOG = logging.getLogger(__name__)
# exit code constants
COMPLETED_OK = 0
ERROR_USER_INPUT = 1
ERROR_STOPING_SERVICES = 2
ERROR_STARTING_SERVICES = 3
ERROR_PATH_NOT_A_DIRECTORY = 4
ERROR_UNHANDLED_EXCEPTION = 5
ERROR_VCHA_ENABLED = 6
ERROR_MISSING_IMPORT = 7
sys.path.append(os.environ['VMWARE_PYTHON_PATH'])
try:
from cis.tools import get_install_parameter
from cis.exceptions import InstallParameterException
except ImportError:
class InstallParameterException(Exception):
"""Imitates missing InstallParameterException class"""
if sys.platform in ['win32', 'cygwin', 'windows']:
try:
from six.moves import winreg
except ImportError:
import _winreg as winreg
try:
import win32security
import win32api
except ImportError:
LOG.error("Unable to import win32security and/or win32api")
sys.exit(ERROR_MISSING_IMPORT)
SCRIPT_VERSION = "1.6.0"
JNDI_PATH = "org/apache/logging/log4j/core/lookup/JndiLookup.class"
BACKUP_DIR = ""  # This is initialized below
LOG_DIR = os.environ.get('VMWARE_LOG_DIR')
LOG_NAME = "vmsa-2021-0028"
HASHING_CHUNK_SIZE = 1024 * 1024
SAFE_SHA256_HASHES = [
'085e0b34e40533015ba6a73e85933472702654e471c32f276e76cffcf7b13869', # log4j-core-2.16.0.jar from apache.org
'5d241620b10e3f1475320bc9552cf7bcfa27eeb9b1b6a891449e76db4b4a02a8'  # log4j-core-2.16.0.jar from mvnrepository/build-artifactory
]
# Deployment types
DEPLOY_TYPE_PSC = "infrastructure"
DEPLOY_TYPE_EMBEDDED = "embedded"
DEPLOY_TYPE_MANAGEMENT = "management"
class Environment:
"""
Collect VMware environment specific information
"""
def __init__(self):
"""
Computes the vCenter version. Use on vCenter 6.5 and later only
"""
self.__gateway = False
LOG.debug("Determining vCenter version and type")
self.__is_windows = sys.platform in ['win32', 'cygwin', 'windows']
if self.__is_windows:
reg = winreg.ConnectRegistry(None, winreg.HKEY_LOCAL_MACHINE)
key = winreg.OpenKey(reg, r"SOFTWAREVMware, Inc.vCenter Server")
self.__build = winreg.QueryValueEx(key, 'BuildNumber')[0]
self.__version = winreg.QueryValueEx(key, 'ProductVersion')[0]
else:
with open("/etc/applmgmt/appliance/update.conf", 'r') as file_descriptor:
data = json.load(file_descriptor)
self.__build = data['build']
with open("/etc/issue", 'r') as file_descriptor:
for line in file_descriptor:
if not line.strip():
continue
version = line
if "Gateway" in line:
self.__gateway = True
break
version = version.rsplit(' ', 1)[1]
self.__version = version.strip()
try:
LOG.debug("Getting deploy type")
self.__deploytype = get_install_parameter('deployment.node.type', quiet=True)
except (InstallParameterException, NameError):
try:
file = os.path.join(os.environ['VMWARE_CFG_DIR'], 'deployment.node.type')
with open(file, 'r') as file_descriptor:
self.__deploytype = file_descriptor.read()
except Exception as e:
LOG.error("Unhandled exception occurred while trying "
"to get system deployment type from configuration file: %s", e)
except Exception as e:
LOG.error("Unhandled exception occurred while trying "
"to get system deployment type using python script: %s", e)
self.__has_vcha = os.path.isfile("/etc/vmware-vcha/vcha.cfg")
LOG.debug("Computed version: %s", str(self))
def __str__(self):
return ("Version: %s; Build: %s; Deployment type: %s; "
"Gateway: %s; VCHA: %s; Windows: %s;"
% (self.__version, self.__build, self.__deploytype,
self.__gateway, self.__has_vcha, self.__is_windows))
def is_7(self):
"""Checks if current environment is version 7.x"""
return self.__version.startswith("7.")
def is_6(self):
"""Checks if current environment is version 6.x"""
return self.__version.startswith("6.")
def is_65(self):
"""Checks if current environment is version 6.5.x"""
return self.__version.startswith("6.5.")
def is_gateway(self):
"""Checks if current environment is gateway"""
return self.__gateway
def has_identity_svcs(self):
"""Checks if current environment has identity services"""
return self.__deploytype in [DEPLOY_TYPE_EMBEDDED, DEPLOY_TYPE_PSC]
def has_mgmt_svcs(self):
"""Checks if current environment has appliance management services"""
return self.__deploytype in [DEPLOY_TYPE_EMBEDDED, DEPLOY_TYPE_MANAGEMENT]
def has_vcha(self):
"""Checks if current environment has HA enabled"""
return self.__has_vcha
def is_windows(self):
"""Checks if current environment is windows OS"""
return self.__is_windows
ENV = Environment()
class Services(object):
"""Helper class for start/stop all services using service-control"""
def __init__(self):
if ENV.is_windows():
self.service_control =
os.path.join(os.environ['VMWARE_CIS_HOME'], 'bin', 'service-control.bat')
else:
self.service_control = "/usr/bin/service-control"
@classmethod
def run_command(cls, cmd):
"""
execute a command with the given input and return the return code and output
"""
LOG.debug("Running command: %s", str(cmd))
# Note: close_fds is always set to False for windows. This is because
# stdin/stdout flags don't work with close_fds on Windows.
process = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE,
close_fds=False)
stdout, stderr = process.communicate()
ret = process.returncode
LOG.debug("Done running command")
if isinstance(stdout, str):
stdout = stdout.decode(sys.getfilesystemencoding())
if isinstance(stderr, str):
stderr = stderr.decode(sys.getfilesystemencoding())
if ret != 0:
LOG.error("RC = %snStdout = %snStderr = %s", ret, stdout, stderr)
return ret, stdout, stderr
def stop(self):
"""
Runs external script which stops system services
"""
LOG.info("stopping services")
cmd = [self.service_control, '--stop', '--all']
ret, stdout, _ = Services.run_command(cmd)
if ret != 0:
LOG.error("error occurred while trying to stop vmware services")
sys.exit(ERROR_STOPING_SERVICES)
LOG.debug(stdout)
def start(self):
"""
Runs external script which start system services
"""
LOG.info("starting services")
cmd = [self.service_control, '--start', '--all']
ret, stdout, _ = Services.run_command(cmd)
if ret != 0:
LOG.error("error occurred while trying to start services")
sys.exit(ERROR_STARTING_SERVICES)
LOG.debug(stdout)
class Log4jCommandFlagPatcher(object):
"""
Contains common functionality for patching log4j flag option in both
linux and windows
"""
def __init__(self, dryrun):
self.dryrun = dryrun
self.cfgdir = os.path.join(os.environ['VMWARE_CFG_DIR'], 'vmware-vmon', 'svcCfgfiles')
self.runtime_dir = os.environ['VMWARE_RUNTIME_DATA_DIR']
self.home = os.environ['VMWARE_CIS_HOME']
self.results = []
self.issues = []
@classmethod
def _workaround_exists(cls, filename):
return Log4jCommandFlagPatcher._file_contains(filename, "formatMsgNoLookups")
@classmethod
def _file_contains(cls, filename, str_val):
LOG.debug("Checking %s for '%s'", filename, str_val)
with open(filename, 'r') as file_descriptor:
data = file_descriptor.read()
if str_val in data:
LOG.debug("Found.")
return True
LOG.debug("Not found.")
return False
def _add_to_results(self, filename):
if filename not in self.results:
self.results.append(filename)
def _patch_file(self, filename, replace_func):
if Log4jCommandFlagPatcher._workaround_exists(filename):
return
LOG.info("Found VULNERABLE FILE: %s", filename)
self._add_to_results(filename)
if self.dryrun:
return
LOG.debug("Patching file: %s", filename)
backup_file(filename)
Log4jCommandFlagPatcher._replace_file_content(
filename,
replace_func)
@classmethod
def _replace_file_content(cls, filename, replace_func):
stat = os.stat(filename)
try:
with open(filename, 'r', encoding='utf-8') as in_file:
content = in_file.read()
except Exception:
with codecs.open(filename, 'r', encoding='utf-8') as in_file:
content = in_file.read()
content = replace_func(content)
try:
with open(filename, 'w', encoding='utf-8') as out_file:
out_file.write(content)
except Exception:
with codecs.open(filename, 'w', encoding='utf-8') as out_file:
out_file.write(content)
set_file_perms(filename, stat)
def patch(self):
"""Main function to run configuration file patching mechanism"""
LOG.error("Unimplemented Base method")
class Log4jCommandFlagWindowsPatcher(Log4jCommandFlagPatcher):
"""
patches vCenter windows specific content
"""
def _patch_wrapper(self, filename):
def patch_replace_func(content):
max_line = 0
lines = content.splitlines()
for line in lines:
if "wrapper.java.additional" not in line:
continue
line = line.split("=", 1)[0]
try:
val = int(line.rsplit(".", 1)[1])
if val > max_line:
max_line = val
except ValueError:
continue
if max_line == 0:
msg = ("Cannot find wrapper.java.additional in the "
"cofiguration file: %s. Skipping modifing configuration." % filename)
self.issues.append(msg)
LOG.error(msg)
return content
latest_param = "wrapper.java.additional.%s=" % max_line
new_param =
"nwrapper.java.additional.%s="-Dlog4j2.formatMsgNoLookups=true"n"
% (max_line + 1)
new_lines = []
for line in lines:
if latest_param in line.replace(" ",""):
line = line + new_param
new_lines.append(line)
return "n".join(new_lines)
self._patch_file(filename, patch_replace_func)
@classmethod
def _contains_startargs(cls, filename):
with open(filename) as file_descriptor:
if re.search(r'StartCommand.*/java', file_descriptor.read()):
LOG.debug("Java vMON file: %s", filename)
return True
LOG.debug("Not a Java vMON file: %s", filename)
return False
def patch_vmon_confs(self):
"""patches vMON configuration"""
LOG.debug("Executing vMON service configurations check")
if not os.path.exists(self.cfgdir):
msg = ("vMON configuration not found: %s."
" Cannot perform automated remediation on this file. " % self.cfgdir)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
return
for cfgfile in [f for f in os.listdir(self.cfgdir) if f.endswith('.json')]:
filename = os.path.join(self.cfgdir, cfgfile)
if Log4jCommandFlagPatcher._workaround_exists(filename)
or not Log4jCommandFlagWindowsPatcher._contains_startargs(filename):
continue
LOG.info("Found VULNERABLE FILE: %s", filename)
self._add_to_results(filename)
if self.dryrun:
continue
backup_file(filename)
def add_log4j_flag(text):
content = json.loads("n".join(line for line in text.splitlines()
if not line.lstrip().startswith("//")))
command = content.get('StartCommand')
if command and (command.endswith('bin/java') or command.endswith('bin/java.exe')):
content['StartCommandArgs'].insert(
0,
'-Dlog4j2.formatMsgNoLookups=true')
return json.dumps(content, sort_keys=True, indent=4)
Log4jCommandFlagPatcher._replace_file_content(filename, add_log4j_flag)
def patch_psc_client(self):
"""patches PSX Client configuration"""
LOG.debug("Executing PSC Client configuration check")
filename = os.path.join(self.runtime_dir, 'vmware-psc-client', 'conf', 'wrapper.conf')
if not os.path.exists(filename):
msg = ("Cannot locate the PSC Client configuration at: %s."
". Cannot perform automated remediation on this file. ", filename)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
return
self._patch_wrapper(filename)
def patch_sts(self):
"""patches Secure Token Services configuration"""
LOG.debug("Executing Secure Token Services (STS) configuration check")
filename = os.path.join(self.runtime_dir, 'VMwareSTSService', 'conf', 'wrapper.conf')
if not os.path.exists(filename):
filename = os.path.join(self.home,'VMware Identity Services', 'wrapper', 'conf', 'wrapper.conf')
if not os.path.exists(filename):
msg = ("Cannot locate the STS configuration at : %s."
"Cannot perform automated remediation." % filename)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
return
self._patch_wrapper(filename)
def patch_perfcharts(self):
"""patches perjcharts configuration"""
LOG.debug("Executing perfcharts check")
filename = os.path.join(self.home, 'perfcharts', 'wrapper', 'conf', 'wrapper.conf')
if not os.path.exists(filename):
msg = ("Cannot locate the perfcharts configuration at: %s."
" Cannot perform automated remediation." % filename)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
return
self._patch_wrapper(filename)
def patch_idmd(self):
"""patches Identity Management Service configuration"""
LOG.debug("Executing Identity Management Service check")
regkey = r'HKEY_LOCAL_MACHINESOFTWAREWow6432NodeApache Software Foundation'
+ r'Procrun 2.0VMwareIdentityMgmtServiceParametersJava'
try:
with winreg.ConnectRegistry(None, winreg.HKEY_LOCAL_MACHINE) as reg:
with winreg.OpenKey(
reg,
r"SOFTWAREWow6432NodeApache Software FoundationProcrun 2.0"
+ r"VMwareIdentityMgmtServiceParametersJava",
0,
winreg.KEY_ALL_ACCESS) as key:
win32security.AdjustTokenPrivileges(
win32security.OpenProcessToken(
win32api.GetCurrentProcess(),
40),
0,
[(win32security.LookupPrivilegeValue(None, 'SeBackupPrivilege'), 2)])
options = winreg.QueryValueEx(key, 'Options')[0]
if "-Dlog4j2.formatMsgNoLookups=true" in options:
# already handled
return
LOG.info("Found a VULNERABLE component:  RegKey %s", regkey)
self._add_to_results(r'RegKey: HKEY_LOCAL_MACHINESOFTWAREWow6432Node'
r'Apache Software FoundationProcrun 2.0'
r'VMwareIdentityMgmtServiceParametersJava')
if self.dryrun:
return
regkeybackuppath = os.path.join(BACKUP_DIR, "IDMD_REG_BACKUP")
winreg.SaveKey(key, regkeybackuppath)
options.append("-Dlog4j2.formatMsgNoLookups=true")
winreg.SetValueEx(key, "Options", 0, winreg.REG_MULTI_SZ, options)
LOG.info("VULNERABLE REGKEY: %s backed up to %s", regkey, regkeybackuppath)
except Exception as e:
msg = ("Cannot perform automated remediation of Identity "
"Management Service configuration. Unhandled exception occurred: %s" % e)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
def patch(self):
"""patches all windows vulnerable configurations"""
self.patch_vmon_confs()
if ENV.has_identity_svcs():
self.patch_sts()
self.patch_idmd()
if ENV.is_65():
self.patch_psc_client()
return self.results, self.issues
class Log4jCommandFlagVCSAPatcher(Log4jCommandFlagPatcher):
"""
Patches vCenter Virtual Appliance specific configuration
"""
def patch_vmon(self):
"""patches vMON service configuration"""
LOG.debug("Executing vMON service configurations check")
filename = "/usr/lib/vmware-vmon/java-wrapper-vmon"
if not os.path.exists(filename):
msg = ("Cannot locate the vMON configuration at: %s."
" Cannot perform automated remediation on this file." % filename)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
return
if Log4jCommandFlagPatcher._workaround_exists(filename):
return
checkstring1 = 'exec $java_start_bin $jvm_dynargs "$@"'
new_config_entries1 = 'log4j_arg="-Dlog4j2.formatMsgNoLookups=true"'
+ "n"
+ 'exec $java_start_bin $jvm_dynargs $log4j_arg "$@"'
checkstring2 = 'exec $java_start_bin $jvm_dynargs $security_dynargs $original_args'
new_config_entries2 = 'log4j_arg="-Dlog4j2.formatMsgNoLookups=true"'
+ "n"
+ 'exec $java_start_bin $jvm_dynargs $log4j_arg $security_dynargs $original_args'
if Log4jCommandFlagPatcher._file_contains(filename, checkstring1):
self._patch_file(
filename,
lambda content: content.replace(checkstring1, new_config_entries1))
elif Log4jCommandFlagPatcher._file_contains(filename, checkstring2):
self._patch_file(
filename,
lambda content: content.replace(checkstring2, new_config_entries2))
else:
msg = ("vMON script exists and vulnerability marker were " +
"not found on file: %s. Please, verify this is expected." % filename)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
def patch_psc_client(self):
"""patches PSC client configuration"""
LOG.debug("Executing PSC Client configuration check")
filename = "/etc/rc.d/init.d/vmware-psc-client"
if not os.path.exists(filename):
msg = ("Cannot locate the PSC Client configuration at: %s."
" Cannot perform automated remediation on this file." % filename)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
return
new_config_entries = '-Dlog4j2.formatMsgNoLookups=true n'
+ '             $DAEMON_CLASS start'
self._patch_file(
filename,
lambda content: content.replace('$DAEMON_CLASS start', new_config_entries))
def patch_sts(self):
"""patches Secure Token Service configuration"""
LOG.debug("Executing Secure Token Service (STS) check")
filename = "/etc/rc.d/init.d/vmware-stsd"
if not os.path.exists(filename):
msg = ("Cannot locate the STS configuration at: %s."
" Cannot perform automated remediation on this file." % filename)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
return
new_config_entries = '-Dlog4j2.formatMsgNoLookups=true n'
+ '            $DAEMON_CLASS start'
self._patch_file(
filename,
lambda content: content.replace('$DAEMON_CLASS start', new_config_entries))
def patch_idmd(self):
"""patches Identity Management Service configuration"""
LOG.debug("Executing Identity Management Service configuration check")
filename = "/etc/rc.d/init.d/vmware-sts-idmd"
if not os.path.exists(filename):
msg = ("Cannot locate the Identity Management Service configuration at: %s."
" Cannot perform automated remediation on this file." % filename)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
return
new_config_entries = '-Dlog4j2.formatMsgNoLookups=true n'
+ '                  $DEBUG_OPTS'
self._patch_file(
filename,
lambda content: content.replace("$DEBUG_OPTS", new_config_entries))
def patch_vum(self):
"""patches Update Manager service configuration"""
LOG.debug("Executing Update Manager Service configuration check")
filename = "/usr/lib/vmware-updatemgr/bin/jetty/start.ini"
if not os.path.exists(filename):
msg = ("Cannot locate the Update Manager Service configuration at: %s."
" Cannot perform automated remediation on this file." % filename)
self.issues.append(msg)
LOG.error("%s Please follow the manual steps described in "
return
if Log4jCommandFlagPatcher._workaround_exists(filename):
return
LOG.info("Found a VULNERABLE FILE: %s", filename)
self._add_to_results(filename)
if self.dryrun:
return
LOG.debug("Patching VUM file: %s", filename)
backup_file(filename)
with open(filename, 'a') as file_descriptor:
file_descriptor.write("-Dlog4j2.formatMsgNoLookups=true")
def patch(self):
"""patches all VC appliance vulnerable configurations"""
self.patch_vmon()
if ENV.has_identity_svcs():
if ENV.is_65():
self.patch_psc_client()
if ENV.is_6():
self.patch_sts()
self.patch_idmd()
if ENV.has_mgmt_svcs() and ENV.is_7() and not ENV.is_gateway():
self.patch_vum()
return self.results, self.issues
def prompt_service_restart(accept_services_restart, start=False):
"""
Prompts user that a service stop and start operations would be required
"""
service_action = Services()
if not start:
user_choice = 'y'
if not accept_services_restart:
try:
user_choice =
raw_input("A service stop and start is required to "
"complete this operation.  Continue?[y]")
except NameError:
user_choice =
input("A service stop and start is required to "
"complete this operation.  Continue?[y]")
LOG.debug("User chose '%s'", user_choice)
else:
LOG.debug("Skipping user choice and assuming stop and start of services")
if user_choice.lower() == 'y':
service_action.stop()
else:
LOG.info("Cannot continue without stopping services. Exiting...")
sys.exit(ERROR_USER_INPUT)
else:
service_action.start()
def patch_configuration(dryrun):
"""
Apply configuration file patching functionality
"""
if ENV.is_windows():
return Log4jCommandFlagWindowsPatcher(dryrun).patch()
return Log4jCommandFlagVCSAPatcher(dryrun).patch()
def setup_logging(log_dir):
"""
Sets logging to write to vmware log system directory, current working directory, and console
"""
def set_handler(handler, loglevel):
formatter = logging.Formatter(
"%(asctime)s %(levelname)s %(funcName)s: %(message)s",
datefmt='%Y-%m-%dT%H:%M:%S')
handler.setFormatter(formatter)
handler.setLevel(loglevel)
LOG.addHandler(handler)
LOG.setLevel(logging.DEBUG)
file_name = LOG_NAME + "_" + datetime.utcnow().strftime("%Y_%m_%d_%H_%M_%S") + '.log'
if log_dir:
# User specified log directory
file_path = os.path.join(log_dir, file_name)
elif LOG_DIR:
# LOG for support bundles
file_path = os.path.join(LOG_DIR, file_name)
else:
# LOG in current working directory
file_path = os.path.join(os.getcwd(), file_name)
file_handler = logging.FileHandler(file_path)
set_handler(file_handler, logging.DEBUG)
# console handler
console_handler = logging.StreamHandler(sys.stdout)
set_handler(console_handler, logging.INFO)
def is_java_archive(filename):
"""
Return true if the given filename ends with either .jar or .war
:type filename: str
:rtype: bool
"""
return filename.lower().endswith((".jar", ".war"))
def find_zip_files(dirnames, parent_type=None, dryrun=False):
"""
Find and process all files under given dirnames to find the offending log4j
class.
:type dirnames: list[str]
:param parent_type: whether the parent of these dirnames come from a .war file.
The value is only "war" or None.
:type parent_type: str
:param dryrun: A boolean to indicate whether the archive should be mitigated.
False means to mitigate it.
:type dryrun: bool
:return: List of filepaths that have been processed
:rtype: list[str]
"""
if not isinstance(dirnames, list):
dirnames = [dirnames]
processed_files = []
for root, _, files in chain.from_iterable(os.walk(path) for path in dirnames):
for file_name in files:
if is_java_archive(file_name):
if not os.path.getsize(os.path.join(root, file_name)) == 0:
processed = process_archive(file_name, root, parent_type, dryrun)
if processed is not None:
processed_files.append(processed)
else:
LOG.debug("File: %s is 0 bytes in size.  Skipping...", file_name)
continue
return processed_files
def has_vulnerable_class(filename):
"""
Returns true if given filepath contains the offending log4j class.
:param filename: name of a zip file to process
:type filename: str
:rtype: bool
"""
with zipfile.ZipFile(filename, 'r') as zip_obj:
# Get list of files names in zip
list_of_files = zip_obj.namelist()
return any(f.endswith(JNDI_PATH) for f in list_of_files)
def is_safe_version(filename):
"""
Returns true if given filepath matches a hash of the known good versions.
:param filename: name of a zip file to check
:type filename: str
:rtype: bool
"""
sha256 = hashlib.sha256()
with open(filename, 'rb') as file_descriptor:
while True:
chunk = file_descriptor.read(HASHING_CHUNK_SIZE)
if not chunk:
break
sha256.update(chunk)
LOG.debug("%s, %s", filename, sha256.hexdigest())
return sha256.hexdigest() in SAFE_SHA256_HASHES
def backup_file(filepath):
"""
Move given filepath to under BACKUP_DIR with preserving the file hierarchy.
:type filepath: str
:return: The backed up filepath under BACKUP_DIR
:rtype: str
"""
normpath = os.path.normpath(filepath)
normpath = os.sep.join(normpath.split(os.sep)[1:])
backuppath = os.path.join(BACKUP_DIR, normpath + ".bak")
# move file to .bak and remove original
try:
if not os.path.exists(os.path.dirname(backuppath)):
os.makedirs(os.path.dirname(backuppath))
copy_file(filepath, backuppath)
set_file_perms(backuppath, os.stat(filepath))
LOG.info("VULNERABLE FILE: %s backed up to %s", filepath, backuppath)
except OSError:
LOG.error("Failed to create backup of %s. Check if backup already exists.", filepath)
return None
return backuppath
def set_file_perms(filepath, stat):
"""
Setting the given stat to the filepath.
:type filepath: str
:type stat: os.stat_result
"""
os.chmod(filepath, stat.st_mode)
if not ENV.is_windows():
os.chown(filepath, stat.st_uid, stat.st_gid)
def create_archive(pathname, directory):
"""
Archive given directory to pathname.
:type pathname: str
:type directory: str
"""
orig_dir = os.getcwd()
os.chdir(directory)
with zipfile.ZipFile(pathname, 'w') as zip_obj:
# Iterate over all the files in directory
for folder_name, _, filenames in os.walk(directory):
for filename in filenames:
# create complete filepath of file in directory
file_path = os.path.join(folder_name, filename)
# Add file to zip
zip_obj.write(file_path, file_path.replace(directory, ''))
os.chdir(orig_dir)
def process_war(filepath, dryrun=False):
"""
This func processes a .war file by following steps:
1. Create a temporary dir
2. Extract the .war to #1
3. Move .war to BACKUP_DIR
4. Find and process the files in #1, so the jars will be off of offending class, if any.
5. Archive the temporary dir #1 back to the original filename
:type filepath: str
:param dryrun: A boolean to indicate whether the archive should be mitigated.
False means to mitigate it.
:type dryrun: bool
:return: None if the war does not contain any offending class and was not changed at all;
the filepath if the war was handled.
"""
results = None
# create a temp dir to extract war file
dirpath = tempfile.mkdtemp()
try:
with zipfile.ZipFile(filepath) as war_zip:
war_zip.extractall(dirpath)
except (zipfile.error, OSError):
LOG.warning("Bad zip file: %s", filepath)
return None
stat = os.stat(filepath)
processed_files = find_zip_files(dirpath, 'war', dryrun)
if len(processed_files) > 0:
LOG.info("Found a VULNERABLE WAR file with: %s", filepath)
results = filepath
if not dryrun:
backuppath = backup_file(filepath)
if backuppath is None:
LOG.error("could not process file %s", filepath)
return None
create_archive(filepath, dirpath)
set_file_perms(filepath, stat)
LOG.info("VULNERABLE FILE: %s backed up to %s", filepath, backuppath)
shutil.rmtree(dirpath, ignore_errors=True)
return results
def process_jar(filepath, parent_type, dryrun):
"""
Processes a .jar file by:
1. Proceedes only archive has vulnerable JNDI class or is not log4j version 2.16
2. Move original file to backup folder
3. Traverses archive files and writes new archive to original file path by skipping JNDI class
4. Mimics permissions from original file
"""
try:
if not has_vulnerable_class(filepath):
return None
if is_safe_version(filepath):
LOG.info("Found a safe version: %s", filepath)
return None
except (zipfile.error, OSError):
LOG.debug("Bad zip file: %s", filepath)
return None
LOG.info("Found a VULNERABLE FILE: %s", filepath)
if dryrun:
# if dryrun, we don't want to change the file, just report it.
return filepath
stat = os.stat(filepath)
backuppath = backup_file(filepath)
if backuppath is None:
LOG.error("could not process file %s", filepath)
return None
zin = zipfile.ZipFile(backuppath, 'r')
zout = zipfile.ZipFile(filepath, 'w')
for item in zin.infolist():
if not item.filename.endswith(JNDI_PATH):
buffer = zin.read(item.filename)
zout.writestr(item, buffer)
set_file_perms(filepath, stat)
zout.close()
zin.close()
# don't keep backups of jar inside war
if parent_type == 'war':
os.remove(backuppath)
else:
LOG.info("VULNERABLE FILE: %s backed up to %s", filepath, backuppath)
return filepath
def process_archive(filename, root, parent_type, dryrun=False):
"""
Processes the filename under root
:type filename: str
:type root: str
:param parent_type: Whether filename comes from a .war, value can only be None or war
:type parent_type: str
:param dryrun: A boolean to indicate whether the archive should be mitigated.
False means to mitigate it.
:type dryrun: bool
:return: filepath if it was handled; None otherwise
"""
filepath = root + os.path.sep + filename
if filepath.endswith('.war'):
return process_war(filepath, dryrun)
return process_jar(filepath, parent_type, dryrun)
def parse_args(args):
"""
Parse arguments
"""
parser = argparse.ArgumentParser(description="""
VMSA-2021-0028 vCenter tool; Version: %s
This tool deletes the JndiLookup.class file from *.jar and *.war
files.
On Windows systems the tool will by default traverse the folders
identified by the VMWARE_CIS_HOME, VMWARE_CFG_DIR, VMWARE_DATA_DIR
and VMWARE_RUNTIME_DATA_DIR variables.
On vCenter Appliances the tool will search by default from the root
of the filesystem.
All modified files are backed up if the process needs to be reversed
due to an error.
""" % SCRIPT_VERSION)
parser.add_argument("-d", "--directories",
nargs="+",
default=[],
help="space separated list of directories to check recursively "
"for CVE-2021-44228 vulnerable java archive files.",
metavar="dirnames")
parser.add_argument("-a", "--accept-services-restart",
action="store_true",
help="accept the restart of the services without having "
"manual prompt confirmation for the same")
parser.add_argument("-r", "--dryrun",
action="store_true",
help="Run the script and log vulnerable files without "
"mitigating them. The vCenter services are not "
"restarted with this option.")
parser.add_argument("-b", "--backup-dir",
help="Specify a backup directory to store original files.")
parser.add_argument("-l", "--log-dir",
help="Specify a directory to store log files.")
return parser.parse_args(args)
def get_dirnames(args):
"""
Gets a list of directories to check for exploit.
If no explicit parameter provided uses root folder for linux
and a list of environment variables provided folders for windows
"""
dirnames = args.directories if args.directories is not None else []
if not dirnames:
if ENV.is_windows():
dirnames = []
for env_var in ['VMWARE_CIS_HOME',
'VMWARE_CFG_DIR',
'VMWARE_DATA_DIR',
'VMWARE_RUNTIME_DATA_DIR']:
val = os.environ.get(env_var)
if val is not None:
dirnames.append(val)
if not dirnames:
dirnames = [os.getcwd()]
else:
dirnames = [os.path.abspath(os.sep)]
for dirname in dirnames:
if not os.path.isdir(dirname):
LOG.debug("Error: provided '%s' path is not a directory", dirname)
sys.exit(ERROR_PATH_NOT_A_DIRECTORY)
return dirnames
def set_backup_dir(args):
"""
sets backup folder location
"""
global BACKUP_DIR
if args.backup_dir:
BACKUP_DIR = args.backup_dir
# make sure BACKUP_DIR exists
if not os.path.exists(BACKUP_DIR):
os.makedirs(BACKUP_DIR)
else:
BACKUP_DIR = tempfile.mkdtemp()
def print_summary(dryrun, zip_processed, flagcheck_processed, issues):
"""
prints summary of script execution result
"""
note = """
NOTE: Running this script again with the --dryrun
flag should now yield 0 vulnerable files.
"""
summary = "n=====     Summary     =====n"
if len(zip_processed) + len(flagcheck_processed) > 0:
if dryrun:
summary += "List of vulnerable java archive files:nn"
else:
summary += "Backup Directory: %sn" % BACKUP_DIR
summary += "List of processed java archive files:nn"
for file_path in zip_processed:
summary += "%sn" % file_path
if dryrun:
summary += "nList of vulnerable configuration files:nn"
else:
summary += "nList of processed configuration files:nn"
for file_path in flagcheck_processed:
summary += "%sn" % file_path
elif len(issues) == 0:
summary += "nNo vulnerable files found!n"
if len(issues) > 0:
summary += "nFound following configuration file issues:nn"
for issue in issues:
summary += "- %sn" % issue
if dryrun:
summary += "nTotal found: %s" % (len(zip_processed) + len(flagcheck_processed))
else:
summary += "nTotal fixed: %s" % (len(zip_processed) + len(flagcheck_processed))
summary += note
summary += "nLog file: %sn" % LOG.handlers[0].baseFilename
summary += "==========================="
LOG.info(summary)
def main(args):
"""
Main function, this function should return an int as return code.
"""
args = parse_args(args)
setup_logging(args.log_dir)
LOG.info("Script version: %s", str(SCRIPT_VERSION))
LOG.info("vCenter type: %s", str(ENV))
if ENV.has_vcha():
LOG.error("You need to remove VCHA to apply the workarounds as"
" mentioned in Impacts section of KB https://kb.vmware.com/s/article/87081n")
sys.exit(ERROR_VCHA_ENABLED)
if not args.dryrun:
prompt_service_restart(args.accept_services_restart)
else:
LOG.info("Running in dryrun mode.")
set_backup_dir(args)
dirnames = get_dirnames(args)
LOG.debug("Inspecting folders: %s", str(dirnames))
zip_processed = find_zip_files(dirnames, None, args.dryrun)
LOG.debug("Found %s vulnerable zip files.", str(len(zip_processed)))
LOG.debug("Running vCenter Configuration checks for log4j2.formatMsgNoLookups flag.")
flagcheck_processed, issues = patch_configuration(args.dryrun)
LOG.debug("Found %s vulnerable configuration files.", str(len(flagcheck_processed)))
print_summary(args.dryrun, zip_processed, flagcheck_processed, issues)
if not args.dryrun:
prompt_service_restart(args.accept_services_restart, start=True)
LOG.info("Done.")
return COMPLETED_OK
if __name__ == '__main__':
try:
sys.exit(main(sys.argv[1:]))
except Exception as e:
LOG.exception("Unhandled exception occurred while running the script: %s", e)
sys.exit(ERROR_UNHANDLED_EXCEPTION)

Optional Arguments Of Script

Optional arguments to the above python script are given below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Optional Arguments
-h, --help            show this help message and exit
-d dirnames [dirnames ...], --directories dirnames [dirnames ...]
space separated list of directories to check
recursively for CVE-2021-44228 vulnerable java archive
files.
-a, --accept-services-restart
acccept the restart of the services without having
manual prompt confirmation for the same
-r, --dryrun          Run the script and log vulnerable files without
mitigating them. The vCenter services are not
restarted with this option.
-b BACKUP_DIR, --backup-dir BACKUP_DIR
Specify a backup directory to store original files.
-l LOG_DIR, --log-dir LOG_DIR
Specify a directory to store log files.

How to roll back an update using yum in CentOS & AlmaLinux

Rollback an update using yum as follows. As an example, we will be using the install and rollback of the screen package.

Note that it is not possible to rollback selinux, selinux-policy-*, kernel, or glibc (dependencies of glibc such as gcc) packages to a previous version. Downgrading a system to a minor version is not recommended as this might leave the system in an undesired state.

  • The screen tool can be installed and uninstalled using yum in the following example.

# yum install screen

  • The next step is to locate the transaction ID for which a ‘undo’ is desired.
  • The transaction ID we’re looking for is ‘8,’ so go ahead and undo the step. Use yum history info 8 before doing the undo if you want to view more details to make sure this is the transaction you want to undo.
  • Enter the following command to undo transaction 8.

# yum history undo 8

It is always recommended to perform a full system backup before installing any update, and yum history is not intended to replace system backups.

It’s always a good idea to document the state of the system before and after patching. This should include using the --orphans, --problems, --dupes, and --leaves flags with package-cleanup.

yum history undo will require access to all earlier RPM versions; as a result, the system must have access to the older RPM versions.

It is suggested that you carefully review the output of package-cleanup --orphans before doing updates to determine which currently installed RPMs are no longer available in the enabled repositories.

Using CentOS’s regular repositories shouldn’t be an issue because several RPM versions are kept in these places.

Check the yum output/logs for any messages and rpmnew, orig, save files created after making any RPM modifications.

ssh connection

SSH connection fails with messages “no hostkey alg”

The SSH connection from CentOS 6 to CentOS 8 fails while running CentOS 8 in FIPS mode.

  • Getting the following ssh debug output:

debug2: mac_setup: found hmac-sha1
debug1: kex: server->client aes128-ctr hmac-sha1 none
debug2: mac_setup: found hmac-sha1
debug1: kex: client->server aes128-ctr hmac-sha1 none
no hostkey alg

  • To resolve this issue, on CentOS 6 you should generate ECDSA host keys with correct permissions as below.

ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -C '' -N ''
chmod 600 /etc/ssh/ssh_host_ecdsa_key
chmod 640 /etc/ssh/ssh_host_ecdsa_key.pub
restorecon /etc/ssh/ssh_host_ecdsa_key.pub

  • To allow ssh clients to accept ECDSA host keys, add the following in /etc/ssh/sshd_config file on the ssh server.

Host <hostname/IP>
Hostkeyalgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521

CentOS 6 servers can connect to CentOS 8 in FIPS mode using ECDSA keys (not created and accepted automatically).

CentOS 8 only supports SHA2 hashes for RSA host keys, hence they are incompatible with CentOS 6’s SHA1-based RSA host keys. However, CentOS 6 computers can connect to CentOS 8 in FIPS mode using ECDSA keys (not produced and accepted automatically).

telnet

How to configure Telnet in Linux

For security reasons, It is advised to use secure SSH rather than Telnet to connect to a server. When you use Telnet, your passwords are sent through the network in plain text. As a result, the root user is not permitted to use Telnet by default.

  • Make sure you have the necessary telnet-server and telnet RPMs installed before using Telnet.

# rpm -qa | grep telnet
telnet-server-3.13-26.EL8
telnet-3.13-26.EL8

  • If you don’t have the telnet-server or telnet packages installed, you can use the RPMs from your installation disk to install them, or you can use the yum program to download and install the package.

# up2date telnet-server telnet

# yum install telnet-server telnet

  • Add the service to firewalld.

# firewall-cmd --add-service=telnet --zone=public

  • If you want to keep this rule permanent, add the –permanent flag to the command.

# firewall-cmd --add-service=telnet --zone=public --permanent

  • Reload the firewall rules.

# firewall-cmd --reload

  • Add the service to SELinux (not required with selinux-policy-3.12.1-77 or later)

# semanage port -a -t telnetd_port_t -p tcp 23

  • Start and Enable the telnet service

# systemctl start telnet.socket
# systemctl enable telnet.socket
or
# systemctl enable telnet.socket --now

  • Test the telnet server.

# telnet server2
Trying 3.3.3.3...
Connected to 3.3.3.3.
Escape character is '^]'.Kernel 3.10.0-327.el7.x86_64 on an x86_64
server2 login: telnet-user
Password:

selinux

Unable to disable SELinux

The SELINUX=disabled option in the /etc/selinux/config file was removed from the kernel with the RHEL9.0 release. The system boots up with SELinux enabled but no policy loaded if SELINUX=disabled is defined in /etc/selinux/config.

  • The only way to turn it off is to run the kernel with the selinux=0 option.
  • Use grubby to ensure that the bootloader always boots with selinux=0.

# grubby --update-kernel ALL --args selinux=0

  • To restore SELinux to its default state, follow these steps.

# grubby --update-kernel ALL --remove-args selinux

Please note support for disabling SELinux through /etc/selinux/config has been removed in CentOS 9, and AlmaLinux 9.

error nginx

nginx: [emerg] socket() [::]:8080 failed (97: Address family not supported by protocol)

When starting nginx, you may get the following error.

nginx: [emerg] socket() [::]:8080 failed (97: Address family not supported by protocol)

This indicates that the server’s IPv6 is disabled, causing the service to fail owing to an unsupported address family.

  • Search for the following directive in the NGINX configuration file to resolve this error.

# listen [::]:80;

  • Replace the above with the following.

listen 80;

nginx

How to configure NGINX to display 404 errors locally

The error_page directive handles nginx problems, whereas proxy_intercept_errors controls whether proxied replies with codes greater than or equal to 300 should be delivered to a client or intercepted and redirected to nginx for processing.

Regardless of the http status code, nginx will return whatever the proxy server returns by default.

proxy_intercept_errors on;
error_page 404 /404.html;

load balancing nginx

Difference between NGINX load balance methods

NGINX Open Source supports four load‑balancing methods.

  • Round Robin
  • Least Connections
  • IP Hash
  • Generic Hash

Round Robin

Requests are spread uniformly across the servers, taking into account server weights. This approach is used by default (there is no way to turn it off).

upstream backend {
# no load balancing method is specified for Round Robin
server backend1.example.com;
server backend2.example.com;
}

Least Connections

A request is delivered to the server with the fewest active connections, again taking into account server weights.

upstream backend {
least_conn;
server backend1.example.com;
server backend2.example.com;
}

IP Hash

The client IP address is used to determine which server a request is forwarded to. In this situation, the hash value is calculated using either the first three octets of the IPv4 address or the entire IPv6 address. Unless the server is unavailable, the approach ensures that requests from the same address are routed to the same server.

upstream backend {
ip_hash;
server backend1.example.com;
server backend2.example.com;
}

If one of the servers has to be withdrawn from the loadbalancing cycle momentarily, it can be designated with the down option to keep the existing hashing of client IP addresses. Requests that this server was supposed to handle are automatically routed to the next server in the group.

upstream backend {
server backend1.example.com;
server backend2.example.com;
server backend3.example.com down;
}

Generic Hash

A user-defined key, which might be a text string, variable, or a combination, determines which server a request is sent to. For instance, the key could be a URI or a coupled source IP address and port, as in this example.

upstream backend {
hash $request_uri consistent;
server backend1.example.com;
server backend2.example.com;
}

The hash directive’s optional consistent parameter enables ketama consistent hash load balancing. Based on the user-defined hashed key value, requests are dispersed uniformly among all upstream servers. Only a few keys are remapped when an upstream server is added to or withdrawn from an upstream group, reducing cache misses in load-balancing cache servers or other applications that store state.

Note: NGINX Plus adds two more methods, but we don’t support them because they’re part of their enterprise package.