[Pkg-samba-maint] [Git][samba-team/samba][master] 3362 commits: VERSION: Bump version up to 4.10.0pre1...

Mathieu Parent gitlab at salsa.debian.org
Tue Sep 10 19:56:35 BST 2019



Mathieu Parent pushed to branch master at Debian Samba Team / samba


Commits:
37d056f4 by Karolin Seeger at 2018-07-12T08:11:01Z
VERSION: Bump version up to 4.10.0pre1...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
710ce1c3 by Karolin Seeger at 2018-07-12T08:15:44Z
WHATSNEW: Start release notes for Samba 4.10.

Signed-off-by: Karolin Seeger <kseeger at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
7b5a47b8 by Stefan Metzmacher at 2018-07-12T12:25:17Z
smbd: add [un]become_guest() helper functions

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
23319ef5 by Stefan Metzmacher at 2018-07-12T12:25:17Z
smbd: add smbd_impersonate_debug_create() helper

This will be used to implement no-op impersonation
for the create_conn_struct_as_root() case were we
don't really have other unrelated events in the loop
and only need a valid tevent wrapper context to avoid
double free on the raw event context on teardown.

This also adds useful debugging instead of being
a full no-op wrapper.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
5285966e by Stefan Metzmacher at 2018-07-12T12:25:17Z
smbd: add simple noop smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers

As a start these are just wrappers arround
smbd_impersonate_debug_create(), without any real impersonation.
But this will change shortly.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
0dcaa070 by Stefan Metzmacher at 2018-07-12T12:25:18Z
smbd: make use of smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers

For now they just add debugging, but that will change shortly.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
1b804f7a by Stefan Metzmacher at 2018-07-12T12:25:18Z
smbd: implement smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers

This makes sure we're doing the correct impersonation for async
requests, which is a requirement to start adding path based
async SMB_VFS calls.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
e37e41b3 by Stefan Metzmacher at 2018-07-12T12:25:18Z
smbd: avoid explicit change_to_user() in defer_rename_done() already done by impersonation

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
d0b1f96f by Stefan Metzmacher at 2018-07-12T12:25:18Z
smbd: remove unused change_to_root_user() from smbd_sig_hup_handler()

This is handled by using the root_ev_ctx in order to register
the signal event.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
c310647e by Ralph Boehme at 2018-07-12T12:25:18Z
smbd: remove unused change_to_root_user() from brl_timeout_fn()

This is handled by using the root_ev_ctx in order to register
the timer event.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
19e4a085 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: correctly handle pthreadpool_tevent_register_ev() failures

It returns errno values instead of setting 'errno'.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
5e723bc6 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: use unsigned for num_idle, num_threads and max_threads

These can't get negative.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
53a9f3ca by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: explicitly use max_thread=unlimited for pthreadpool_tevent_init() tests

Currently 0 also means unlimited, but that will change soon,
to force no thread and strict sync processing.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
65faef95 by Stefan Metzmacher at 2018-07-12T12:25:18Z
s3:messages: explicitly use max_thread=unlimited for pthreadpool_tevent_init() in messaging_dgm_init()

Currently 0 also means unlimited, but that will change soon,
to force no thread and strict sync processing.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
03830a32 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: consitently use unlock_res for pthread_mutex_unlock() in pthreadpool_add_job()

This makes further restructuring easier to implement and understand.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
e45d33e9 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: use strict sync processing only with max_threads=0

Otherwise it's an error if not at least one thread is possible.

This gives a much saner behaviour and doesn't end up with
unexpected sync processing.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
c9f54db1 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: use talloc_zero() in tests_cmocka.c setup_pthreadpool_tevent()

This was found with valgrind.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f1dac71a by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: fix helgrind error in pthreadpool_free()

We need to pthread_mutex_lock/unlock the pool mutex
before we can destroy it.

The following test would trigger this.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
76474a6f by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: expand test_create() to check unlimited, sync and one thread pool

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
505d298e by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: add pthreadpool_max_threads() and pthreadpool_queued_jobs() helpers

These can be used to implement some kind of flow control in the caller.
E.g. unless pthreadpool_queued_jobs() is lower than
pthreadpool_max_threads() is good to prepare new jobs.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
e4dfd3da by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: add pthreadpool_tevent_max_threads() and pthreadpool_tevent_queued_jobs()

These can be used to implement some kind of flow control in the caller.
E.g. unless pthreadpool_tevent_queued_jobs() is lower than
pthreadpool_tevent_max_threads() is good to prepare new jobs.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
4e54543b by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: add pthreadpool_cancel_job()

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
59768416 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: don't process further jobs when shutting down

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f19552e2 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: split out a pthreadpool_stop() from pthreadpool_destroy()

This can be used in combination with pthreadpool_cancel_job() to
implement a multi step shutdown of the pool.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
cdbad904 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: let pthreadpool_tevent_job_send() fail with an invalid pool

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
245d684d by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: split out pthreadpool_tevent_job from pthreadpool_tevent_job_state

This makes it much easier to handle orphaned jobs,
we either wait for the immediate tevent to trigger
or we just keep leaking the memory.

The next commits will improve this further.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
791c0514 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: add pthreadpool_tevent_job_cancel()

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
fa070d90 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: make use of pthreadpool_stop() in pthreadpool_tevent_destructor()

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
25756425 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: maintain a global list of orphaned pthreadpool_tevent_jobs

Instead of leaking the memory forever, we retry the cleanup,
if other pthreadpool_tevent_*() functions are used.

pthreadpool_tevent_cleanup_orphaned_jobs() could also be called
by external callers.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
3eee52b4 by Stefan Metzmacher at 2018-07-12T15:18:00Z
pthreadpool: allocate glue->tctx on glue as memory context.

This means it will go aways together with glue and thte event context.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Thu Jul 12 17:18:01 CEST 2018 on sn-devel-144

- - - - -
ce9ac51e by David Mulder at 2018-07-12T20:11:22Z
gpo: Fix asserts in gpo testing

These tests weren't using python's unit testing
asserts.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2494e338 by David Mulder at 2018-07-12T20:11:22Z
gpo: Disable python3 testing

The gpo module doesn't work in python3 yet,
causing this test to fail on python3.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13525

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4c7348e4 by David Mulder at 2018-07-12T20:11:22Z
python: Allow forced signing via smb.SMB()

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
57faf35c by David Mulder at 2018-07-12T20:11:22Z
gpo: Read GPO versions locally, not from sysvol

Non-kdc clients cannot read directly from the
sysvol, so we need to store the GPT.INI file
locally to read each gpo version.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2412c704 by David Mulder at 2018-07-12T20:11:23Z
gpo: Offline policy application via cache

Read policy files from the cache, rather than
the sysvol. This enables offline policy apply.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
46629f53 by David Mulder at 2018-07-12T20:11:23Z
param: Add python binding for lpcfg_state_path

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
387dc358 by David Mulder at 2018-07-12T20:11:23Z
gpo: add register_gp_extension for registering gp extensions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b4f09af2 by David Mulder at 2018-07-12T20:11:23Z
gpo: add unregister_gp_extension for unregistering gp extensions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4436b670 by David Mulder at 2018-07-12T20:11:23Z
gpo: add list_gp_extensions for listing registered gp extensions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fb6c250b by David Mulder at 2018-07-12T20:11:23Z
gpo: Tests for gp_ext register/unregister

Adds testing for the gp_ext register and
unregister functions, as well as testing
the list function.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
126b7892 by David Mulder at 2018-07-12T20:11:23Z
gpo: Dynamically load gp_exts

This loads Group Policy Client Side Extensions
similar to the way that they are loaded on a
Windows client. Extensions are installed to a
configuration file in the samba cache path where
they receive a unique GUID matched with the path
to the python gp_ext file. Classes which inherit
from the gp_ext class (as defined in gpclass.py)
will be dynamically loaded.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
de8b30a7 by David Mulder at 2018-07-12T20:11:23Z
gpo: Add user policy extensions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b05b6198 by David Mulder at 2018-07-12T20:11:23Z
gpo: Don't duplicate guids in the apply log

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f3358f04 by David Mulder at 2018-07-12T23:05:22Z
gpo: Specify samba module when importing from gpclass

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Jul 13 01:05:22 CEST 2018 on sn-devel-144

- - - - -
9b164637 by Joe Guo at 2018-07-12T23:12:24Z
samdb: use int for get and set methods instead of digit str

This will make the API work in a nature way.

Also, because of a defect in ldb API, code like `res[0]["maxPwdAge"][0]`
will return bytes even in Python3, which will cause trouble. By casting
the value to int, we avoid the str/bytes issue.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5b96e349 by Noel Power at 2018-07-12T23:12:24Z
python/samba/tests: Py2/Py3 allow import of ndr_(un)pack to work

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d5cd9af7 by Noel Power at 2018-07-12T23:12:24Z
python/samba/tests: Py2/Py3 port for hexdump

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7d435711 by Noel Power at 2018-07-12T23:12:24Z
python/samba/emulate: Fix some more missed exception tuple assignments

In python3 we need to change

    except LdbError as e:
-        (status, _) = e
to
    except LdbError as e:
+        (status, _) = e.args

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
337ae8cd by Noel Power at 2018-07-12T23:12:24Z
python/samba/tests: remove Py2 specific imports.

Remove some python2 specific import, probably this was due to
previous unavailability for some c-modules in python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7b170206 by Noel Power at 2018-07-12T23:12:24Z
lib/ldb: Implement a bytes derived object for attributes py2/py3

ldb attributes are either bytes (py3) or str (py2)

Some places in the code do str(res[0]['attribute'][0])
which results in
   'result' (py2)
  b'result' (py3)

or more commonly the attribute is used to construct a string e.g.
   "blah=" + res[0]['attribute'][0] + ",foo,bar=...."

giving
   "blah=result,foo,bar=...." (py2)
and very unhelpfully
   "blah=b'result',foo,bar=...." (py3)

lots of code already constructs various strings for passing to other
api using the above. To avoid many excessive
    res[0]['attribute'][0].decode('utf8')

code like 'res[0]['attribute'][0]'

will now return LdbBytes (a new object subclassing 'bytes') in py3
instead of bytes. This object has a custom '__str__' method which
attempts to return a string decoded to uft8. In Py2 this will behave as
it did previously (this is the safer option at the moment)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e915d18e by Noel Power at 2018-07-12T23:12:24Z
s4/torture/drs/python: Py2/Py2 fix tab/space also incorrect unicode usage

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
352eee57 by Noel Power at 2018-07-12T23:12:24Z
python/samba/netcmd: fix py2/py3 bytes usage for replace

base64.b64encode returns bytes in py3 make sure associated replace
uses 'b' for strings passed to replace

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
c46fd168 by Noel Power at 2018-07-12T23:12:24Z
python/samba/netcmd: Protect variable that can be None

In py3 None variable cannot be compared with '>' '<' etc operators

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
bbc5d314 by Noel Power at 2018-07-12T23:12:24Z
python/samba/netcmd: Fix relative module import

Part of future changes needed to enable samba4.drs.replica_sync_rodc
for PY3.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
883c3d8d by Noel Power at 2018-07-12T23:12:24Z
s4/librpc/ndr: allow GUID to accept unicode also

This needed since _GUID_string method change
(in source4/torture/drs/python/drs_base.py) which makes use use
a unicode guid at times now

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
2a2c2213 by Noel Power at 2018-07-12T23:12:25Z
s4/dsdb/tests/python: base64.b64encode returns bytes

adjust to unicode for py2/py3 compat needed as part of changes
to ensure samba4.ldap.password_settings will work with PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
343dc311 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs: ndr_upack needs bytes in py3 (samba4.drs.repl_move)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
06d3eac8 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: py2/py3 port map / ord usage

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
0e930dfb by Noel Power at 2018-07-12T23:12:25Z
python/samba: Add cmp_fn and cmp_to_key_fn functions for py2/py3

the cmp function and the cmp paramater (e.g. to sort functions)
no longer exist in python3.

cmp_fn is provides the missing functionality of the py2 cmp builtin
function.

cmp_to_key_fn allows the key paramater (e.g. for sort) to use the
old py2 cmp function for sorting. Note: the cmp_to_key is present in
since 2.7 (hence the inclusion of the source code for this function pre
that version)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>

- - - - -
77ee94c3 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: use cmp_fn and key=cmp_to_key_fn for py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
409ad5c0 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: use cmp_fn for cmp (for py2/py3 compat)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
08c9ce41 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: long is not used in py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fee61c57 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: xrange -> range for py2/py3 compat

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
f59a20eb by Noel Power at 2018-07-12T23:12:25Z
s4/torure/drs/python: Fix incorrect use of unicode which doesn't exist in py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1a9786fe by Noel Power at 2018-07-12T23:12:25Z
python/samba/tests: various py3 porting for ord/chr

various messages are lists of ints that need converting to str or bytes
depending on py2/py3, others are str/bytes that need modification and
are converted to lists or string char or ints for modificate (and then
reconstructed as str/bytes again)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d96d85a2 by Noel Power at 2018-07-12T23:12:25Z
python/samba/kcc: md5 needs to be passed bytes in py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
f8d97513 by Noel Power at 2018-07-13T02:01:59Z
python/samba/netcmd: Fix wrong exception referenced in code

post commit: 52729d35495db638c84caa8cc6f5ffdf0b670353 wrong
exception name was referenced.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Jul 13 04:01:59 CEST 2018 on sn-devel-144

- - - - -
c406a5be by Björn Jacke at 2018-07-13T09:44:02Z
docs: mention that the echo handler is for SMB1 only

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Karolin Seeger <kseeger at samba.org>

Autobuild-User(master): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(master): Fri Jul 13 11:44:02 CEST 2018 on sn-devel-144

- - - - -
1c8d1cce by Jeremy Allison at 2018-07-13T09:45:42Z
s3: torture: Test SMB1 cli_splice() fallback path when doing a non-full file splice.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13527

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>

- - - - -
c9656fd2 by Jeremy Allison at 2018-07-13T12:57:14Z
s3: libsmbclient: Fix cli_splice() fallback when reading less than a complete file.

We were always asking for SPLICE_BLOCK_SIZE even when the
remaining bytes we wanted were smaller than that. This works
when using cli_splice() on a complete file, as the cli_read()
terminated the read at the right place. We always have the
space to read SPLICE_BLOCK_SIZE bytes so this isn't an overflow.

Found by Bailey Berro <baileyberro at google.com>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13527

Signed-off-by: Bailey Berro <baileyberro at google.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>

Autobuild-User(master): David Disseldorp <ddiss at samba.org>
Autobuild-Date(master): Fri Jul 13 14:57:14 CEST 2018 on sn-devel-144

- - - - -
4e123c46 by Alexander Bokovoy at 2018-07-13T15:45:35Z
wafsamba/samba_abi: always hide ABI symbols which must be local

binutils 2.31 is going to change how shared libraries are linked, such
that they always provide their own local definitions of the _end, _edata
and _bss_start symbols.  This would all be fine, except for shared
libraries that export all symbols be default.  (Rather than just
exporting those symbols that form part of their API).

According to binutils developers, we should only export the symbols we
explicitly want to be used. We don't use this principle for all our
libraries and deliberately don't want to have ABI versioning control for
all of them, so the change I introduce here is to explicitly mark those
symbols that will always be added by default linker configuration with
binutils 2.31 as local. Right now these are '_end', '_edata', and
'__bss_start' symbols.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Jul 13 17:45:36 CEST 2018 on sn-devel-144

- - - - -
49d6c3f0 by David Disseldorp at 2018-07-13T22:14:13Z
s3: torture: adjust SMB1 cli_splice() test sizes

The test writes 20M and then splices just over 1M (13M is intended,
but there's a 1024*0124 typo). Fix the type and reduce the size of the
dataset to make it run faster - cli_splice works with 1M chunks
(SPLICE_BLOCK_SIZE), and the reproducer only requires that the splice
size is not chunk-aligned.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Jul 14 00:14:13 CEST 2018 on sn-devel-144

- - - - -
26fd7096 by Günther Deschner at 2018-07-14T01:09:00Z
s3-tldap: do not install test_tldap

Guenther

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13529

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Jul 14 03:09:00 CEST 2018 on sn-devel-144

- - - - -
8cb96438 by David Disseldorp at 2018-07-16T19:11:30Z
dbwrap: determine basename once instead of three times

Currently determined twice in the clear-if-first codepath and once in
the ctdb code path. Do it once at the top of the function.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Mon Jul 16 21:11:30 CEST 2018 on sn-devel-144

- - - - -
3761d42e by Ralph Boehme at 2018-07-17T11:33:06Z
tevent: fix CID 1437976 dereference before null check

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
ba991da4 by Ralph Boehme at 2018-07-17T14:21:21Z
tevent: fix CID 1437974 dereference after null check

Probably a copy/paste error from the tevent_debug() statement a few
lines above as at this place we want to pass main_ev directly to
tevent_debug() anyway.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Jul 17 16:21:21 CEST 2018 on sn-devel-144

- - - - -
72531974 by Ralph Boehme at 2018-07-17T22:04:10Z
s4: torture: run test_durable_v2_open_reopen2_lease() in a subdirectory

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13535

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e60e9368 by Ralph Boehme at 2018-07-17T22:04:10Z
s3: smbd: fix path check in smbd_smb2_create_durable_lease_check()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13535

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2e65a126 by Volker Lendecke at 2018-07-17T22:04:10Z
kcc: Fix the 32-bit build

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7b0c1f8f by Volker Lendecke at 2018-07-18T01:12:02Z
dsdb: Fix the 32-bit build

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Jul 18 03:12:02 CEST 2018 on sn-devel-144

- - - - -
da115efd by Martin Schwenke at 2018-07-18T09:51:14Z
ctdb-tests: Simplify pstree output in eventd unit tests

pstree truncates output when it exceeds a maximum width - the default
is 132 columns.  A couple of recent
commits (12fd8d7a5c5d14d403aac6cd9e318afcd0a8e159,
b23f3f996038626f618c5b5aa552686c1b852f44) lengthened the command
string in the output so that it is more likely to exceed this limit
and be truncated, as below:

==================================================
Running "cat /memdisk/autobuild/fl/b1851760/ctdb/ctdb/tests/var/eventd/debug_script.log"
--------------------------------------------------
Output (Exit status: 0):
--------------------------------------------------
02.enabled.scri,PID /memdisk/autobuild/fl/b1851760/ctdb/ctdb/tests/var/eventd/events/random/02.enabled.script ...
  `-sleep,PID 99
01.disabled          DISABLED
02.enabled           TIMEDOUT   DATETIME
  OUTPUT: Sleeping for 99 seconds
--------------------------------------------------
Required output (Exit status: 0):
--------------------------------------------------
02.enabled.scri,PID /memdisk/autobuild/fl/b1851760/ctdb/ctdb/tests/var/eventd/events/random/02.enabled.script verbosetimeout
  `-sleep,PID 99
01.disabled          DISABLED
02.enabled           TIMEDOUT   DATETIME
  OUTPUT: Sleeping for 99 seconds

FAILED

It isn't clear that the above example exceeds 132 characters, given
that the PID has been filtered into a fixed string, but it certainly
goes close.  Whether or not it is truncated probably depends on the
width of the PID in the unfiltered output.  This would explain why the
test flaps.

Avoid the output truncation by dropping the -a and -p options to
simplify the pstree output.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13531

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
359e5218 by Martin Schwenke at 2018-07-18T12:53:39Z
ctdb-tests: Loosen match against pstree output in simple test

As per previous commit, pstree can truncate output if it gets too
wide.  Instead of matching against the script's full path and
arguments, just match against the script name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13531

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Wed Jul 18 14:53:39 CEST 2018 on sn-devel-144

- - - - -
0d3aec18 by Gary Lockyer at 2018-07-19T23:47:20Z
dns wildcards: tests to confirm BUG 13536

DNS wildcard matching failing if more than one label to the left of the
wildcard. This commits adds tests to confirm the bug.

Wildcard entry: *.example.org
bar.example.com matches
foo.bar.example.com does not, but it it should.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
cef1b31c by Gary Lockyer at 2018-07-20T02:40:31Z
dns wildcards: fix BUG 13536

The current position in the dns name was not advanced past the '.'
character

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Jul 20 04:40:31 CEST 2018 on sn-devel-144

- - - - -
809967b3 by Jeremy Allison at 2018-07-20T10:25:16Z
s3: smbd: Fix Linux sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
16a58707 by Jeremy Allison at 2018-07-20T10:25:16Z
s3: smbd: Fix Solaris sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
d222caa4 by Jeremy Allison at 2018-07-20T10:25:16Z
s3: smbd: Fix HPUX sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
456e520a by Jeremy Allison at 2018-07-20T10:25:16Z
s3: smbd: Fix FreeBSD sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
582ce5d6 by Jeremy Allison at 2018-07-20T13:14:24Z
s3: smbd: Fix AIX sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Jul 20 15:14:24 CEST 2018 on sn-devel-144

- - - - -
2114768a by Alexander Bokovoy at 2018-07-23T14:24:10Z
s4-dns_server: Only build dns server Python code for AD DC

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
1faaeb00 by Alexander Bokovoy at 2018-07-23T14:24:10Z
s4-dsdb: only build dsdb Python modules for AD DC

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
6de9d878 by Alexander Bokovoy at 2018-07-23T17:36:56Z
python/samba/tests: make sure samba.tests can be imported without SamDB

We are using samba.tests Python module __init__.py file as a catch-all
for all types of helpers. Some of these helpers are only usable with
Samba AD DC targets.

When SamDB is not available in a non-Samba AD DC target, provide a
dummy replacement that simply returns None. This allows to complete
initialization for non-Samba AD DC target tests which do not use
connect_samdb() helper.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

Autobuild-User(master): Günther Deschner <gd at samba.org>
Autobuild-Date(master): Mon Jul 23 19:36:56 CEST 2018 on sn-devel-144

- - - - -
062b518c by Stefan Metzmacher at 2018-07-24T04:55:23Z
librpc: add binding handle support for [smb1]

This will be used to force smb1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
17b12a9b by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: split out smb_raw_negotiate_fill_transport()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
b7e99c25 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: add smbcli_transport_raw_init()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
87d73397 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: use talloc_zero() for struct smb_composite_connect in fetchfile.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
2b68f9b8 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: allow passing an already negotiated connection to smb_composite_connect()

It will just do the session setup and tree connect steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
ce2248c4 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: add smb2_transport_raw_init()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
5ad5b81b by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: split out smb2_connect_session_start()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
f20e607c by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: allow passing an already negotiated connection to smb2_connect_send()

It will just do the session setup and tree connect steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
ca000d89 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: add fallback_to_anonymous to smb2_connect_send()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
5188454b by Stefan Metzmacher at 2018-07-24T04:55:24Z
s4:libcli: allow a fallback to NTLMSSP if SPNEGO is not supported locally

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
e4910f35 by Stefan Metzmacher at 2018-07-24T04:55:24Z
s4:libcli: add smb_connect_nego_{send,recv}()

This can be used to create a connection up to a negotiated
smbXcli_conn.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
70a9cf9c by Alexander Bokovoy at 2018-07-24T04:55:24Z
tests/auth_log: Permit SMB2 service description if empty binding is used for kerberos authentication

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>

- - - - -
802e43bf by Stefan Metzmacher at 2018-07-24T04:55:24Z
python/tests: use explicit "client ipc max protocol = NT1" for samba.tests.net_join_no_spnego

The tests rely on SMB1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
4422f738 by Stefan Metzmacher at 2018-07-24T04:55:24Z
s4:librpc: autonegotiate SMB1/2/3

Windows Server 1709 defaults to SMB2 and does not have SMB1 enabled.
When establishing trust, samba-tool does not specify SMB protocol
version and fail by default.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
6800077c by Stefan Metzmacher at 2018-07-24T04:55:24Z
s3:selftest: run rpc.lsa.lookupsids also with explicit [smb1] and [smb2]

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
c3907288 by Alexander Bokovoy at 2018-07-24T07:55:23Z
samba-tool trust: support discovery via netr_GetDcName

In case a remote DC does not support netr_DsRGetDCNameEx2(),
use netr_GetDcName() instead.

This should help with FreeIPA where embedded smbd runs as a domain
controller but does not implement full Active Directory compatibility.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13538

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Alexander Bokovoy <ab at samba.org>
Autobuild-Date(master): Tue Jul 24 09:55:23 CEST 2018 on sn-devel-144

- - - - -
9d31bbf9 by Stefan Metzmacher at 2018-07-24T15:38:25Z
tevent: use talloc_zero_size() for the private state in tevent_context_wrapper_create()

This is watch tevent_req_create() uses and what callers of
tevent_context_wrapper_create() would therefore also expect.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
26b35cb2 by Stefan Metzmacher at 2018-07-24T15:38:26Z
pthreadpool: make sure a pthreadpool is marked as stopped in child processes

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
617d9c87 by Stefan Metzmacher at 2018-07-24T15:38:26Z
pthreadpool: test pthreadpool_tevent_max_threads() returns the expected result

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
c51cae58 by Stefan Metzmacher at 2018-07-24T15:38:26Z
pthreadpool: replace assert_return_code(ret, 0); with assert_int_equal(ret, 0);

We need to assert the exact value!

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
2ebb5847 by Stefan Metzmacher at 2018-07-24T15:38:26Z
lib/replace: check for __thread support

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
6f8c1b67 by Stefan Metzmacher at 2018-07-24T15:38:26Z
third_party/*_wrapper/wscript: remove redundant configure checks

HAVE___THREAD and HAVE_DESTRUCTOR_ATTRIBUTE are already checked
as part of Samba.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
5fa5764f by Stefan Metzmacher at 2018-07-24T15:38:26Z
replace: add checks for atomic_thread_fence(memory_order_seq_cst) and add possible fallbacks

This implements a full memory barrier.
On ubuntu amd64 with results in an 'mfence' instruction.

This is required to syncronization between threads, where
there's typically only one write of a memory that should be
synced between all threads with the barrier.

Much more details can be found here:
https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/_005f_005fatomic-Builtins.html#g_t_005f_005fatomic-Builtins
https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/_005f_005fsync-Builtins.html#g_t_005f_005fsync-Builtins

The main one we use seems to be in C11 via stdatomic.h,
the oldest fallback is __sync_synchronize(), which is available
since 2005 in gcc.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9656b8d8 by Stefan Metzmacher at 2018-07-24T15:38:26Z
pthreadpool: add some lockless coordination between the main and job threads

In the direction from the main process to the job thread, we have:

- 'maycancel', which is set when tevent_req_cancel() is called,
- 'orphaned' is the job request, tevent_context or pthreadpool_tevent
  was talloc_free'ed.

The job function can consume these by using:

   /*
    * return true - if tevent_req_cancel() was called.
    */
   bool pthreadpool_tevent_current_job_canceled(void);

   /*
    * return true - if talloc_free() was called on the job request,
    * tevent_context or pthreadpool_tevent.
    */
   bool pthreadpool_tevent_current_job_orphaned(void);

   /*
    * return true if canceled and orphaned are both false.
    */
   bool pthreadpool_tevent_current_job_continue(void);

In the other direction we remember the following points
in the job execution:

- 'started'  - set when the job is picked up by a worker thread
- 'executed' - set once the job function returned.
- 'finished' - set when pthreadpool_tevent_job_signal() is entered
- 'dropped'  - set when pthreadpool_tevent_job_signal() leaves with orphaned
- 'signaled' - set when pthreadpool_tevent_job_signal() leaves normal

There're only one side writing each element,
either the main process or the job thread.

This means we can do the coordination with a full memory
barrier using atomic_thread_fence(memory_order_seq_cst).
lib/replace provides fallbacks if C11 stdatomic.h is not available.

A real pthreadpool requires pthread and atomic_thread_fence() (or an
replacement) to be available, otherwise we only have pthreadpool_sync.c.
But this should not make a real difference, as at least
__sync_synchronize() is availabe since 2005 in gcc.
We also require __thread which is available since 2002.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
30c97da7 by Stefan Metzmacher at 2018-07-24T15:38:26Z
s3:wscript: don't check for valgrind related headers twice

We already check them in lib/replace/wscript.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
66aaa22f by Stefan Metzmacher at 2018-07-24T15:38:27Z
lib/replace: also check for valgrind/helgrind.h

This will be used in lib/pthreadpool/pthreadpool_tevent.c
in order to avoid extected helgrind/drd warnings.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9b73fda9 by Stefan Metzmacher at 2018-07-24T15:38:27Z
pthreadpool: add helgrind magic to PTHREAD_TEVENT_JOB_THREAD_FENCE_*()

This avoids the expected helgrind/drd warnings on the job states which
are protected by the thread fence.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
aa9b64ec by Stefan Metzmacher at 2018-07-24T15:38:27Z
pthreadpool: maintain a list of job_states on each pthreadpool_tevent_glue

We should avoid traversing a linked list within a thread without holding
a mutex!

Using a mutex would be very tricky as we'll likely deadlock with
the mutexes at the raw pthreadpool layer.

So we use somekind of spinlock using atomic_thread_fence in order to
protect the access to job->state->glue->{tctx,ev} in
pthreadpool_tevent_job_signal().

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f23cac39 by Stefan Metzmacher at 2018-07-24T15:38:27Z
pthreadpool: add a comment about a further optimization in pthreadpool_tevent_job_destructor()

This seems to be a really rare race, it's likely that the immediate
event will still trigger and cleanup.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
40d15260 by Ralph Boehme at 2018-07-24T15:38:27Z
pthreadpool: test cancelling and freeing pending pthreadpool_tevent jobs/pools

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Ralph Boehme <slow at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>

- - - - -
7cb27238 by Ralph Boehme at 2018-07-24T15:38:27Z
configure: check for Linux specific unshare() with CLONE_FS

Note we still need some kind of runtime detection as
it can fail in some constraint container setups, which
reject the whole unshare() syscall instead of just the
once used for container features.

In case unshare(CLONE_FS) works, we can have a per thread
current working directory and use [f]chdir() safely in
worker threads.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
65e4742d by Ralph Boehme at 2018-07-24T15:38:27Z
pthreadpool: call unshare(CLONE_FS) if available

This paves the way for pthreadpool jobs that are path based.

Callers can use pthreadpool_per_thread_cwd() to check if
the current pool supports it.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
12a45ee1 by Stefan Metzmacher at 2018-07-24T15:38:27Z
pthreadpool: add pthreadpool_tevent_[current_job_]per_thread_cwd()

This can be used to check if worker threads run with
unshare(CLONE_FS).

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
fbafdc99 by Stefan Metzmacher at 2018-07-24T15:38:28Z
pthreadpool: add tests for pthreadpool_tevent_[current_job_]per_thread_cwd()

Note this currently this doesn't enforce the support for
unshare(CLONE_FS) as some contraint container environment
(e.g. docker) reject the whole unshare() system call.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
3c4cdb29 by Stefan Metzmacher at 2018-07-24T15:38:28Z
pthreadpool: add pthreadpool_restart_check[_monitor_{fd,drain}]()

This makes it possible to monitor the pthreadpool for exited worker
threads and may restart new threads from the main thread again.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f9745d8b by Stefan Metzmacher at 2018-07-24T15:38:28Z
pthreadpool: implement pthreadpool_tevent_wrapper_create() infrastructure

This can be used implement a generic per thread impersonation
for thread pools.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
fb6b6cf3 by Ralph Boehme at 2018-07-24T15:38:28Z
pthreadpool: test cancelling and freeing jobs of a wrapped pthreadpool_tevent

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
0d2eeb94 by Ralph Boehme at 2018-07-24T15:38:28Z
lib/util: rename USE_LINUX_THREAD_CREDENTIALS to HAVE_LINUX_THREAD_CREDENTIALS

The define reflects the results of a feature test, not a configure
option.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
10fc65b7 by Stefan Metzmacher at 2018-07-24T18:35:17Z
s3:util_sec: add a cache to set_thread_credentials()

Calling set_thread_credentials() with the same values,
skips syscalls the 2nd time.

We only do this if '__thread' is supported to provide
thread local storage.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Tue Jul 24 20:35:17 CEST 2018 on sn-devel-144

- - - - -
6a91ba4f by Volker Lendecke at 2018-07-24T18:36:49Z
lib: Remove an #include "includes.h"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a845e961 by Volker Lendecke at 2018-07-24T18:36:49Z
lib: Remove an #include "includes.h"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7daf5e62 by Volker Lendecke at 2018-07-24T18:36:50Z
lib: Fix prototype of srprs_str

Many callers use "-1" as the "len" argument. That's what ssize_t is for.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3c9b88ba by Volker Lendecke at 2018-07-24T18:36:50Z
nsswitch: Remove IRIX support

According to wikipedia, IRIX has seen the last patch update in August 2006. As
of now, www.sgi.com is unreachable. Probably this code has not been built in
years. If someone wants to revive it, it can be found in the git history.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d4c6b009 by Volker Lendecke at 2018-07-24T18:36:50Z
nsswitch: Make two functions static

nss_irix was the only external user

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
da179b1e by Volker Lendecke at 2018-07-24T18:36:50Z
nsswitch: Correct users of "ctx->is_privileged"

winbindd_context->is_privileged is a bool

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8a530433 by Volker Lendecke at 2018-07-24T18:36:50Z
winbind: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7d40f607 by Volker Lendecke at 2018-07-24T21:31:43Z
winbind: Move variable declarations close to their use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Jul 24 23:31:43 CEST 2018 on sn-devel-144

- - - - -
010bbe53 by Ralph Boehme at 2018-07-24T22:23:12Z
autobuild: add some basic tests for the all static build

This makes sure each module is at least loaded once
and registers itself as a module.

It means that the skel_opaque and skel_transparent vfs examples
are loaded.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
b294c7c6 by Ralph Boehme at 2018-07-24T22:23:12Z
examples/VFS/skel_opaque: add missing audit_file_fn

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
d1633532 by Ralph Boehme at 2018-07-24T22:23:12Z
examples/VFS/skel_opaque: add missing durable handle functions

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
f9db9aee by Stefan Metzmacher at 2018-07-24T22:23:12Z
examples/VFS/skel_opaque: call smb_vfs_assert_all_fns()

This template should always implement all calls.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
68b8e5a5 by Ralph Boehme at 2018-07-24T22:23:12Z
examples/VFS/skel_transparent: add missing audit_file_fn

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
829fdf10 by Ralph Boehme at 2018-07-24T22:23:12Z
examples/VFS/skel_transparent: add missing durable handle functions

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
42e99ec3 by Stefan Metzmacher at 2018-07-24T22:23:12Z
examples/VFS/skel_transparent: call smb_vfs_assert_all_fns()

This template should always include all calls.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
1bc92d10 by Stefan Metzmacher at 2018-07-24T22:23:12Z
vfs_default: call smb_vfs_assert_all_fns()

This module needs to implement every call.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
7d1de8bd by Ralph Boehme at 2018-07-24T22:23:12Z
s3: lib/xattr_tdb: fix listing xattrs

If there's no record in the xattr.tdb, dbwrap_fetch() will return
NT_STATUS_NOT_FOUND. That should not result in an error in callers of
xattr_tdb_load_attrs().

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
76c68bc2 by Ralph Boehme at 2018-07-24T22:23:13Z
s4: libcli/smb2: calculate correct credit charge for finds

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
580ff206 by Ralph Boehme at 2018-07-24T22:23:13Z
s3: libsmb: use smb2cli_conn_max_trans_size() in cli_smb2_list()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
27bb2cbc by Ralph Boehme at 2018-07-24T22:23:13Z
vfs_default: fix async fsync idle/busy time profiling

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
83f01b02 by Ralph Boehme at 2018-07-24T22:23:13Z
s3: vfs: add missing tevent_req_received() to SMB_VFS_PREAD_RECV()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
d769e9ea by Ralph Boehme at 2018-07-24T22:23:13Z
s3: vfs: add missing tevent_req_received() to SMB_VFS_PWRITE_RECV()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
55097f7d by Ralph Boehme at 2018-07-24T22:23:13Z
s3: vfs: add missing tevent_req_received() to SMB_VFS_FSYNC_RECV()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
b2ae22a3 by Ralph Boehme at 2018-07-24T22:23:13Z
s3: vfs: bump to version 39, Samba 4.9 will ship with that

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
1e78cb57 by Ralph Boehme at 2018-07-25T01:23:44Z
s3: vfs: bump to version 40, Samba 4.10 will ship with that

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Wed Jul 25 03:23:44 CEST 2018 on sn-devel-144

- - - - -
7f1de9ff by Jeremy Allison at 2018-07-25T01:24:39Z
s3: smbd: SGI IRIX is officially dead. Remove the kernel oplock code for IRIX.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>

- - - - -
e2ebfd8e by David Disseldorp at 2018-07-25T04:28:21Z
docs/kerneloplocks: drop Irix references

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Jul 25 06:28:21 CEST 2018 on sn-devel-144

- - - - -
79f494e5 by Gary Lockyer at 2018-07-25T04:29:50Z
json: Modify API to use return codes

Modify the auditing JSON API to return a response code, as the consensus
was that the existing error handling was aesthetically displeasing.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6f4f8c51 by Gary Lockyer at 2018-07-25T04:29:50Z
json: Add unit tests for error handling

Add cmocka unit tests to exercise the error handling in the JSON
routines.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a5172ee7 by Gary Lockyer at 2018-07-25T04:29:50Z
dsdb audit logging: remove HAVE_JANSSON from audit_log

This modules is ADDC only and JANSSON is required for the ADDC builds,
so the ifdef is no longer necessary.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
77fa3408 by Gary Lockyer at 2018-07-25T04:29:50Z
dsdb group auditing: remove HAVE_JANSSON from group_audit

This modules is ADDC only and JANSSON is required for the ADDC builds,
so the ifdef is no longer necessary.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6a5346f1 by Gary Lockyer at 2018-07-25T04:29:50Z
dsdb group_audit_test: Remove redundant mocking code

Remove a place holder test and unused mocking code.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a5e02f72 by Gary Lockyer at 2018-07-25T07:28:31Z
lib audit_logging: add _WARN_UNUSED_RESULT_

Have the compiler issue a warning when the return code from the API is
ignored.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Wed Jul 25 09:28:31 CEST 2018 on sn-devel-144

- - - - -
3fd1a41f by Ralph Boehme at 2018-07-25T15:49:05Z
pthreadpool: add a missing include

Reported-by: David Disseldorp <ddiss at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
5cb94ca0 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: only pass struct smbXsrv_client to smb1srv_tcon_disconnect_all()

That's all it needs.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
98487273 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: only pass struct smbXsrv_client to smbXsrv_session_logoff_all()

That's all it needs.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
8d4792e6 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: disconnect/destroy all connections before calling smbXsrv_session_logoff_all()

This means the pending requests are destroyed before tevent_context
impersonation wrapper are destroyed.

Otherwise, with a pending struct tevent_req that is a child of
client->xconn like this in exit_server_common():

conn[ipv4:127.0.0.11:40745] reason[NT_STATUS_CONNECTION_RESET] at ../source3/smbd/smb2_server.c:4015
full talloc report on 'struct smbXsrv_connection' (total   6085 bytes in  43 blocks)
    struct smbd_smb2_request       contains    648 bytes in   1 blocks (ref 0) 0x55ed41634740
    struct smbd_smb2_request       contains   3438 bytes in  32 blocks (ref 0) 0x55ed416331e0
        struct tevent_req              contains   1824 bytes in  20 blocks (ref 0) 0x55ed41635860
            struct smb_filename            contains    206 bytes in   2 blocks (ref 0) 0x55ed41635560
                lease_v2_complex2.dat          contains     22 bytes in   1 blocks (ref 0) 0x55ed4161b950
            struct smbd_smb2_create_state  contains   1386 bytes in  16 blocks (ref 0) 0x55ed41635a10
                struct deferred_open_record    contains    804 bytes in  12 blocks (ref 0) 0x55ed41633090
                    struct defer_open_state        contains    764 bytes in  11 blocks (ref 0) 0x55ed41634f10
                        struct tevent_req              contains    748 bytes in  10 blocks (ref 0) 0x55ed41636390
                            struct tevent_timer            contains     96 bytes in   1 blocks (ref 0) 0x55ed41636ad0
                            struct dbwrap_watched_watch_state contains    420 bytes in   7 blocks (ref 0) 0x55ed41636540
                                struct tevent_req              contains    296 bytes in   5 blocks (ref 0) 0x55ed41636700
                                    struct messaging_filtered_read_state contains     64 bytes in   3 blocks (ref 0) 0x55ed416368b0
                                        struct messaging_dgm_fde       contains      8 bytes in   2 blocks (ref 0) 0x55ed41636950
                                            reference to: struct messaging_dgm_fde_ev

we crash when freeing the xconn as the the tevent_req child has a
cleanup function that tries to access a (wrapped) tevent context that
was already destroyed via smb1srv_tcon_disconnect_all() for SMB1 or
smbXsrv_session_logoff_all() for SMB2 a few lines above.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
0e900d60 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: add missing DO_PROFILE_INC(disconnect) to smbd_server_connection_terminate_ex()

For multi channel connections we should increment the disconnect count
also if we're not closing the last channel.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
2be7518a by Ralph Boehme at 2018-07-25T15:49:06Z
smbd: rename sconn->pool to sconn->raw_thread_pool

This should in future not be used directly, we'll provide
wrapper pools, which will provide impersonation for
path based async calls.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0c972263 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: introduce sconn->sync_thread_pool

This just simulates a threadpool, but executes the
job functions inline (blocking) in the main thread.

This will be used to work arround some OS limitations,
e.g. if per thread credentials or per thread working directory
are not supported.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
1251a536 by Ralph Boehme at 2018-07-25T15:49:06Z
s3: vfs: add smb_vfs_ev_glue

This adds VFS helper functions and that work on a struct smb_vfs_ev_glue
object which bundles two event contexts and a few threadpools.

This will be used to streamline the use of impersonating wrappers
in the SMB_VFS.

Notice the verbose comments in source3/smbd/vfs.c.

This will allow us to introduce path based async operations
to the SMB_VFS layer.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
2dd95c1c by Ralph Boehme at 2018-07-25T15:49:06Z
s3: vfs: add user_vfs_evg to connection_struct

This will be used to in order to pass down the
impersonation magic from the SMB layer through
the SMB_VFS layer.

This includes the following options:

smbd:force sync user path safe threadpool
smbd:force sync user chdir safe threadpool
smbd:force sync root path safe threadpool
smbd:force sync root chdir safe threadpool

They can be used in order to test the non linux code
path on linux, once we get code that makes full use
of the new infrastructure.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
cd37badc by Ralph Boehme at 2018-07-25T15:49:07Z
vfs_aio_pthread: use event context and threadpool from user_vfs_evg

Or the root glue in case we're already root.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
bd79564a by Stefan Metzmacher at 2018-07-25T15:49:07Z
s3:modules: add vfs_not_implemented module

This provides helper functions, which can be used by other modules,
if they don't implement a specific function.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
53d78225 by Ralph Boehme at 2018-07-25T15:49:07Z
examples/VFS/skel_opaque: fix a likely a copy/paste error

This line was probably copied over from skel_transparent.c, remove it.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
f37f8cca by Ralph Boehme at 2018-07-25T15:49:07Z
examples/VFS/skel_opaque: make vfs_fn_pointers static

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
5ab0b4af by Stefan Metzmacher at 2018-07-25T18:44:12Z
examples/VFS/skel_transparent: make vfs_fn_pointers static

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Wed Jul 25 20:44:12 CEST 2018 on sn-devel-144

- - - - -
4e711d18 by Ralph Boehme at 2018-07-25T23:29:38Z
smbd: don't client->connections without checking client != NULL first in exit_server_common()

exit_server_common() can be called also in smbd processes without a
smbXsrv_client structure, e.g. the parent or some background tasks.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Thu Jul 26 01:29:38 CEST 2018 on sn-devel-144

- - - - -
fa9e6a50 by Volker Lendecke at 2018-07-26T20:44:24Z
idmap: Make pointer initialization explicit

Took me a few seconds to find this NULL initializer

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a3b42db9 by Volker Lendecke at 2018-07-26T20:44:25Z
popt: popt 1.16 needs -liconv

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3bff0494 by Volker Lendecke at 2018-07-26T20:44:25Z
smbd: Pass "share_mode_data" to share_entry_forall callback

Quite a bit of the contents have been passed explicitly anyway.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6ed119ec by Volker Lendecke at 2018-07-26T20:44:25Z
smbstatus: Use share_mode_data->leases

This is the only user of share_mode_entry->lease

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2dffcde4 by Volker Lendecke at 2018-07-26T23:42:31Z
smbd: Remove "share_mode_entry->lease"

smbstatus was the only user, and this could be solved by adapting
share_entry_forall.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Jul 27 01:42:31 CEST 2018 on sn-devel-144

- - - - -
5dd84bf5 by Martin Schwenke at 2018-07-27T03:45:20Z
ctdb-common: Fix compilation issue with strncpy()

When configured with --picky-developer and using -O3 with gcc 8.1:

../common/system_socket.c: In function ‘parse_ip_mask’:
../common/system_socket.c:229:2: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
  strncpy(s, str, len+1);
  ^~~~~~~~~~~~~~~~~~~~~~
../common/system_socket.c:223:8: note: length computed here
  len = strlen(str);
        ^~~~~~~~~~~

Use strlcpy() instead and check the result.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13545

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
3b56f200 by Martin Schwenke at 2018-07-27T03:45:20Z
ctdb-protocol: Fix compilation issue with strncpy()

When configured with --picky-developer and using -O3 with gcc 8.1:

../protocol/protocol_util.c: In function ‘ctdb_sock_addr_from_string’:
../protocol/protocol_util.c:282:2: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
  strncpy(s, str, len+1);
  ^~~~~~~~~~~~~~~~~~~~~~
../protocol/protocol_util.c:277:8: note: length computed here
  len = strlen(str);
        ^~~~~~~~~~~

Use strlcpy() instead and check the result.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13545

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4a1fb729 by Martin Schwenke at 2018-07-27T03:45:20Z
ctdb-protocol: Add function ctdb_sock_addr_mask_from_string()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
efaa7690 by Martin Schwenke at 2018-07-27T03:45:21Z
ctdb-tools: Switch to using ctdb_sock_addr_mask_from_string()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
b1fed7e1 by Martin Schwenke at 2018-07-27T03:45:21Z
ctdb-daemon: Switch to using ctdb_sock_addr_mask_from_string()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
77242e76 by Martin Schwenke at 2018-07-27T03:45:21Z
ctdb-common: Drop function parse_ip_mask() and supporting functions

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e7ef0c97 by Martin Schwenke at 2018-07-27T06:42:20Z
ctdb-common: Drop unused function mkdir_p_or_die()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Fri Jul 27 08:42:20 CEST 2018 on sn-devel-144

- - - - -
591d72f9 by Stefan Metzmacher at 2018-07-27T11:07:14Z
ldb_mdb: #ifdef EBADE as it is not portable

E.g. FreeBSD 11.2 doesn't have it.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
ff863f2d by Stefan Metzmacher at 2018-07-27T11:07:14Z
pthreadpool: we need to use pthreadpool_tevent_per_thread_cwd() on the callers pool

In pthreadpool_tevent_job_send() we remember if the job will be chdir
safe. It means we means we need to ask the callers pool when calling
pthreadpool_tevent_per_thread_cwd(), as the callers pool might
be a wrapper using pthreadpool_tevent_force_per_thread_cwd().

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
7e1f9c9d by Ralph Boehme at 2018-07-27T11:07:14Z
s3: vfs: add SMB_VFS_GETXATTRAT_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
c1c8a1f6 by Ralph Boehme at 2018-07-27T11:07:14Z
vfs_default: implement SMB_VFS_GETXATTRAT_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8a6013f6 by Ralph Boehme at 2018-07-27T11:07:14Z
vfs_xattr_tdb: implement SMB_VFS_GETXATTRAT_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
47d77432 by Ralph Boehme at 2018-07-27T11:07:14Z
s3: vfs: add SMB_VFS_GET_DOS_ATTRIBUTES_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0d1fb6a7 by Ralph Boehme at 2018-07-27T11:07:14Z
smbd: split out public parse_dos_attribute_blob() from get_ea_dos_attribute()

In preperation of adding an async version of get_ea_dos_attribute() that
will then call parse_dos_attribute_blob() too.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
1265b516 by Ralph Boehme at 2018-07-27T11:07:14Z
vfs_default: implement SMB_VFS_GET_DOS_ATTRIBUTES_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0feef168 by Ralph Boehme at 2018-07-27T11:07:14Z
smbd: factor out dosmode post processing

We apply some post processing to the dosmode returned from the VFS
function. Move this to a seperate function which will be reused in the
next commit in the async dos_mode_send/recv post processing.

Best viewed with: git show --histogram

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0fbeac03 by Ralph Boehme at 2018-07-27T11:07:14Z
smbd: add dos_mode_at_send/recv()

Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
8de5018c by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: add "get_dosmode" argument to smbd_dirptr_lanman2_entry()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
2a2294bb by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: pass get_dosmode to smbd_dirptr_get_entry()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8954085a by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: pass get_dosmode to mode_fn in smbd_dirptr_get_entry()

This finally uses "get_dosmode" as passed in from the SMB2 layer, but
all callers still pass true, so no change in behaviour.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
bcfc830c by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: rework error exit in smbd_dirptr_lanman2_entry()

The next commit will add code that must be run if status is NT_STATUS_OK
or STATUS_MORE_ENTRIES.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
abef6ca7 by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: factor out smb2_query_directory_next_entry() from smbd_smb2_query_directory_send()

This paves the way for adding async functions into the enumeration loop.

Best viewed with: git show -w

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0e1d11ee by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: fix a long line in smb2_query_directory_next_entry()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
00a26ac9 by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: deal with fsp->aio_requests in close_directory()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8884036b by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: let smbd_dirptr_lanman2_entry return smb_fname

Note that smb_fname is relative to fsp, not conn!

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0736fdcd by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: use async dos_mode_at_send in smbd_smb2_query_directory_send()

Finally: use the new dos_mode_at_send() in the directory enumeration
loop. This means that fetching the DOS attributes for directory entries
is done asynchronously with regard to the enumeration loop.

As the DOS attribute is typically read from an extended attribute in the
filesytem, this avoids sequentially blocking on IO. If the IO subsystem
is slow servicing these request, enabling async processing can result in
performance improvements.

A parametric option

  smbd:async dosmode = true | false (default: false)

can be used to enable the new async processing.

Simulating slow IO with usleep(5000) in the synchronous and asynchronous
versions of SMB_VFS_GET_DOS_ATTRIBUTES(), the results of enumerating a
directory with 10,000 files are:

    smbd:async dosmode = no:

        $ time bin/smbclient -U slow%x //localhost/test -c "ls dir\*" > /dev/null
        real    0m59.597s
        user    0m0.024s
        sys     0m0.012s

    smbd:async dosmode = yes:

        $ time bin/smbclient -U slow%x //localhost/test -c "ls dir\*" > /dev/null
        real    0m0.698s
        user    0m0.038s
        sys     0m0.025s

Performance gains in real world workloads depends on whether the actual
IO requests can be merged and parallelized by the kernel. Without such
wins at the IO layer, the async processing may even be slower then the
sync processing due to the additional overhead.

The following parameters can be used to adapt async processing behaviour
for specific workloads and systems:

        aio max threads = X (default: 100)
        smbd:max async dosmode = Y (default: "aio max threads" * 2)

By default we have at most twice the number of async requests in flight
as threads provided by the underlying threadpool. This ensures a worker
thread that finishes a job can directly pick up a new one without going
to sleep.

It may be advisable to reduce the number of threads to avoid scheduling
overhead while also increasing "smbd:max async dosmode".

Note that we disable async processing for certain VFS modules in the VFS
connect function to avoid the overhead of triggering the sync fallback
in dos_mode_at_send(). This is done for VFS modules that implement the
sync SMB_VFS_GET_DOS_ATTRIBUTES(), but not the async version (gpfs), and
for VFS modules that don't share a real filesystem where fchdir() can be
used (ceph, gluster). It is disabled for catia, because we realized that
the catia name translation macros used on
fsps (CATIA_FETCH_FSP_[PRE|POST]_NEXT) have a bug (#13547).

We use threadpool = smb_vfs_ev_glue_tp_chdir_safe() and then
pthreadpool_tevent_max_threads(threadpool) to get the number of maximum
worker threads which matches the pool used by the low level
SMB_VFS_GETXATTRAT_[SEND|RECV] implementation in vfs_default.

This is a terrible abstraction leak that should be removed in the future
by maybe making it possible to ask a VFS function which threadpool it
uses, internally suporting chaining so VFS function FOO that internally
uses BAR can forward the question to BAR.

On a hyphotetical system that had a getxattrat(dirfd, path, ...)
syscall and at the same time doesn't support per-thread current working
directories (eg FreeBSD doesn't have the latter) but has support for
per-thread-credentials, pthreadpool_tevent_max_threads() on the
tp_chdir_safe threadpool returns 1.

So when hooking the hyphotetical getxattrat() into the async
SMB_VFS_GETXATTRAT_[SEND|RECV] implementation in an VFS module, the
implementation could use the tp_path_safe threadpool, but the SMB2
layer would use the wrong threadpool in the call to
pthreadpool_tevent_max_threads(), resulting in no parallelism.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0b32efcd by Ralph Boehme at 2018-07-27T11:07:15Z
s4: torture: test closing dir handle with in-flight find

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
cbb7eb1f by Ralph Boehme at 2018-07-27T11:07:15Z
selftest: set "smbd:async dosmode = no" in the vfs_aio_pthread share

This just explicitly sets the current default, to ensure the tests that
use this share always use the same "smbd:async dosmode" setting even if
the default changes in the future.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
409d462f by Ralph Boehme at 2018-07-27T14:04:02Z
selftest: run smbtorture3 SMB2-BASIC tests against additional shares

This runs the smbtorture3 SMB2-BASIC and smb2.compound_find tests against shares
with "smbd:async dosmode" enabled.

On the vfs_aio_pthread_async_dosmode_force_sync* shares we
force a sync threadpool which ensures we test behaviour on systems that
don't support unshare(CLONE_FS) and also don't support
per-thread-credentials. This simulates the code path of non linux
systems. And makes sure that we don't regress there.

We also test with xattr_tdb and without.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Ralph Boehme <slow at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Fri Jul 27 16:04:02 CEST 2018 on sn-devel-144

- - - - -
100fe9e9 by Amitay Isaacs at 2018-07-27T22:25:29Z
popt: Check for headers only if building in-tree version

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
156ac244 by Amitay Isaacs at 2018-07-27T22:25:29Z
popt: Add check for iconv library

On glibc based systems, there is no separate iconv library.  Adding a
dependency without checking for the library breaks build on glibc based
systems.

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2ba5fb20 by Andrew Bartlett at 2018-07-28T01:39:48Z
autobuild: Test with and without building bundled popt

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Jul 28 03:39:48 CEST 2018 on sn-devel-144

- - - - -
73298ac8 by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tools: Improve portability by not using /bin/bash directly

FreeBSD and others do not have /bin/bash, so use "/usr/bin/env bash"
for better flexibility.

There are still many integration tests that use /bin/bash but this at
least lets FreeBSD start running tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
dd9d8a20 by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Improve portability by not using /bin/bash directly

FreeBSD and others do not have /bin/bash, so use "/usr/bin/env bash"
for better flexibility.

There are still many integration tests that use /bin/bash but this at
least lets FreeBSD start running tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4a39bc4a by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tools: Avoid use of non-portable getopt in onnode

getopt is being used with non-portable options.  Use simpler,
POSIX-compliant getopts instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
56ffca3e by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Avoid use of non-portable getopt in run_tests.sh

getopt is being used with non-portable options.  Use simpler,
POSIX-compliant getopts instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
896c77df by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Avoid use of non-portable getopt in stubs

getopt is being used with non-portable options.  In most cases use
simpler, POSIX-compliant getopts instead.

In the case of the ctdb test stub command, options can appear after
other arguments, so this requires an additional nested loop.

In the case of smnotify, there are no short options, so handle the
long options manually.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2f2c35a1 by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Improve portability by not using mktemp --tmpdir option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f13824b2 by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Switch some test stubs to use /bin/sh

They don't use any bash features.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
af8c31ea by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-tests: Add errno matching utility

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
e8a1b3db by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-tests: Add required_error() to match on error codes

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
a42a7232 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-common: Switch to ETIMEDOUT from ETIME

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c8756ec1 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-event: Switch to ETIMEDOUT instead of ETIME

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
b886a95e by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-daemon: Switch to using ETIMEDOUT instead of ETIME

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
e1236a85 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-client: Switch to ETIMEDOUT instead of ETIME

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
b7dbe9f3 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-tests: Add ps output filter for freebsd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
b0028dd5 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-tests: Add signal code matching utility

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
23952c91 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Use sigcode to match signals

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
0273171c by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Porting tests should ignore unsupported features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c7041b0f by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-common: Add line based I/O

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c9b42d27 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-protocol: Avoid fgets in ctdb_connection_list_read

C library buffering API can behave in unexpected fashion if underlying
fd for stdin, stdout or stderr is closed and re-opened.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
3bf753e8 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-common: Add fd argument to ctdb_connection_list_read()

This makes testing easier.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
4152e98c by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Do not try to match pstree output in eventd tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
68542dbb by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Simplify pattern matching for ctime output

On freebsd, sed does not accept multiple pattern strings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
96d5c7de by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-scripts: date "+%N" is non-portable

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
07844c2e by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Use portable wc -c instead of stat -c "%s"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
22c3078c by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Replace md5sum with posix cksum

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
65cc36f2 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Use errcode to translate ETIMEDOUT

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
6f5ed2b8 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Fix a typo

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
3047202c by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Strip all spaces from od output

On freebsd, there are trailing spaces in od output.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
be43e080 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-common: Fix the TCP packet length check

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
a44e6987 by Martin Schwenke at 2018-07-28T05:26:24Z
ctdb-docs: Replace obsolete reference to CTDB_DEBUG_HUNG_SCRIPT option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13546

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Sat Jul 28 07:26:24 CEST 2018 on sn-devel-144

- - - - -
56e248de by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-event: Fix "ctdb event status" usage message

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a7a4ee43 by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-common: Factor out basic script abstraction

Provides for listing of scripts and chmod enable/disable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
82e62488 by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-common: Use script abstraction in run_event

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
295826f1 by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-event: Change event-tool script enable/disable to chmod file directly

They no longer go over the socket to eventd to enable and disable
scripts.  Use the event script abstraction to chmod them directly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
5017325c by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-event: Implement event tool "script list" command

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e3ce1a2d by Martin Schwenke at 2018-07-28T18:03:52Z
ctdb-docs: Update documentation for "ctdb event" command

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Sat Jul 28 20:03:52 CEST 2018 on sn-devel-144

- - - - -
d881f0c8 by Justin Stephenson at 2018-07-30T05:34:11Z
s3:libads: Add net ads leave keep-account option

Add the ability to leave the domain with --keep-account argument to avoid
removal of the host machine account.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13498

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
8025467b by Justin Stephenson at 2018-07-30T08:22:59Z
s3:libads: Add net ads keep-account test

Add test for the new --keep-account net ads leave operation

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13498

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Jul 30 10:22:59 CEST 2018 on sn-devel-144

- - - - -
4fcbaae5 by Martin Schwenke at 2018-07-30T12:30:06Z
ctdb-doc: Provide an example script for migrating old configuration

Include an example ctdbd.conf-style file for testing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13550

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Mon Jul 30 14:30:06 CEST 2018 on sn-devel-144

- - - - -
1c8ea099 by Gary Lockyer at 2018-07-30T12:31:52Z
lib ldb: Rename functions to ldb_kv

Rename the ldb key value functions from ltdb_* to ldb_kv_*. The renaming
is preparation for the separation of the tdb specific code from the key
value code.  This work is a follow on from the addition of the lmdb
backend.

Note that the next commit tidies up the code formatting.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9e629466 by Gary Lockyer at 2018-07-30T12:31:52Z
lib ldb: fix formatting of ldb_kv rename.

Clean up the code format after the rename in the previous commit.
Hopefully doing a rename commit followed by a reformat commit makes the
code easier to review.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e8aa764e by Gary Lockyer at 2018-07-30T12:31:52Z
lib ldb: rename struct ltdb_reindex_context

Rename struct ltdb_reindex_context to ldb_kv_reindex_context, as this is
a key value level structure and not a tdb specific structure.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e969de07 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: reformat ltdb_reindex_context rename

Fix up the formatting after the rename of ltdb_reindex_context to
ldb_kv_reindex_context.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c782d710 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename ltdb_context to ldb_kv_context

Rename ltdb_context to ldb_kv_context as it is a key value level
structure and not tdb specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
17a84d12 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename ltdb_req_spy to ldb_kv_req_spy

Rename ltdb_req_spy to ldb_kv_req_spy, as it is key value level and not
tdb specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
641b38e4 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: format rename of ltdb_req_spy

Fix up the code formatting after the rename of ltdb_req_spy to
ldb_kv_req_spy

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e0186d1f by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename ltdb_private to ldb_kv_private

Rename ltdb_private to ldb_kv_private as it contains key value operation
context.

Note there is still some tdb specific context that can be refactored into a
separate structure along the lines of the lmdb context.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b8c9c305 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: format rename ldb_kv_private

Tidy up the code format after the rename of ltdb_private to
ldb_kv_private

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
72724f75 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename ltdb_cache to ldb_kv_cache

Rename ltdb_cache to ldb_kv_cache as it's key value level and not tdb
specific

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f6d5cf5e by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename tdb_key_ctx to key_ctx

Rename tdb_key_ctx to key_ctx, as it's key value level and not tdb
specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d3bfd374 by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: rename ltdb_idxptr to ldb_kv_idxptr

Rename ltdb_idxptr to ldb_kv_idxptr as it's key value level and not tdb
specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9191d3ba by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: remove unused function prototypes

Remove unused function prototypes

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
67c05540 by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: rename ltdb_parse_data_unpack_ctx

Rename ltdb_parse_data_unpack_ctx to ldb_kv_parse_data_unpack_ctx, as
it's a key value level structure and not ltdb specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
19be0be2 by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: move key value code to lib/ldb/ldb_key_value

Move the key value code to a separate subdirectory.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f2d5c2c5 by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: rename LTDB_* constants to LDB_KV_*

Rename all the LTDB_* constants to LDB_KV_* as they are key value level
constants and not tdb specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c891df42 by Gary Lockyer at 2018-07-30T15:23:22Z
lib ldb key value: convert TDB_DATA structs to ldb_val

Convert the key value functions to use ldb_val instead of TDB_DATA.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Jul 30 17:23:22 CEST 2018 on sn-devel-144

- - - - -
80c9219d by Anoop C S at 2018-07-31T02:23:47Z
s3/locking: Fix assertion check on lock reference count

lock_ref_count will always hold the old value prior to change. Thus it
would mean that if lock_ref_count is 0 the new value is already -1 which
is not expected here. Therefore it is better to make sure that it is
always greater than 0 rather than >= 0.

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Jul 31 04:23:47 CEST 2018 on sn-devel-144

- - - - -
6da0d68f by Stefan Metzmacher at 2018-07-31T09:26:16Z
pthreadpool: ignore the return value of poll(NULL, 0UL, 1)

Otherwise Coverity reports this:

CID 1438160:    (CHECKED_RETURN)
Calling "poll(NULL, 0UL, 1)" without checking return value. This
library function may fail and return an error code.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
f68b5ee7 by Stefan Metzmacher at 2018-07-31T09:26:16Z
pthreadpool: reset monitor_fd after calling tevent_fd_set_auto_close()

This tries to convince Coverity that we don't have a resource leak:

CID 1438157:    (RESOURCE_LEAK)
Handle variable "monitor_fd" going out of scope leaks the handle.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
8dac16e8 by Stefan Metzmacher at 2018-07-31T12:20:49Z
smb2_query_directory: make 'return true' explicit in smb2_query_directory_next_entry()

'return req' should do the same as 'return true' for a bool function,
it's implicitly expanded as 'return (req!=NULL)?true:false.

There's no point in that as 'req' is always a valid pointer.

This was most likely just a copy and paste bug.

So we make this explicit now and avoid that Coverity reports this:

CID 1438158:  Null pointer dereferences  (REVERSE_INULL)
Null-checking "req" suggests that it may be null, but it has already
been dereferenced on all paths leading to the check.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Jul 31 14:20:49 CEST 2018 on sn-devel-144

- - - - -
2bfb9b40 by Noel Power at 2018-07-31T14:56:24Z
s3/lib: Fix misleading typo in debug message

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b9340824 by Noel Power at 2018-07-31T14:56:24Z
s3/libsmb: Avoid potential smbpanic calling parse_user_quota_list.

Calling parse_user_quota_list with a NULL buffer can cause a panic, while
this shouldn't happen, I managed to trigger this with an early implementation
of SMB2 quota support in smbd which didn't pass back NT_STATUS_NO_MORE_ENTRIES
when handling a SMB2_0_INFO_QUOTA GETINFO message.
OTHOH the Windows client handled the same situation gracefully.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1033dd9e by Noel Power at 2018-07-31T14:56:24Z
s3/smbd: Don't stat when doing a quota operation (as it's a fake file)

calling SMB_VFS_STAT on the quota fake file fails and caused
FS_INFO/FileFsControlInfo request to error out early, in turn stopped a
Win8.1 client from proceeding with quota queries.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fd7c6f9b by Noel Power at 2018-07-31T14:56:25Z
librpc/idl Add some query [getset]info quota related structures

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
07e91132 by Noel Power at 2018-07-31T14:56:25Z
s3/libsmb: adjust smb1 cli code to use idl structs and ndr push/pull funcs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dd21b4fc by Noel Power at 2018-07-31T14:56:25Z
s3/libsmb: adjust smb2 code for new idl structs & generated ndr push/pull funcs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b8802e27 by Noel Power at 2018-07-31T14:56:25Z
s3/smbd: adjust smb1 server to use idl structs and generated ndr push/pull funcs

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
90703841 by Noel Power at 2018-07-31T14:56:25Z
s3/smbd: smb2 server implementation for query get/set info.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
90da2e59 by Noel Power at 2018-07-31T14:56:25Z
s3/script/test: modify existing smbcquota test to use SMB2 in addition to SMB1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
801c1856 by Noel Power at 2018-07-31T14:56:25Z
s3/script/tests: Add simple (smb1 & smb2) get/set/list tests for smbcquotas

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
705086d8 by Noel Power at 2018-07-31T14:56:25Z
s3/smbd: allow set quota for non root user (when built with --enable-selftest)

Currently it appears you need to be root to set quotas, for test purposes
this requirement needs to be relaxed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e053aad4 by Noel Power at 2018-07-31T17:45:59Z
s3/utils: fix regression where specifying -Unetbios/root works

Usually you need to be root on a linux server to modify quotas. Even
with a linux server joined to a windows AD you could always log in as
local root with smbcquotas. However in recent builds this has changed.
This patch fixes this

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Tue Jul 31 19:45:59 CEST 2018 on sn-devel-144

- - - - -
33d012c3 by Volker Lendecke at 2018-08-03T06:24:06Z
ctdb: Fix a cut&paste error

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13554

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f7b2e5ee by Amitay Isaacs at 2018-08-03T09:14:01Z
ctdb-eventd: Fix CID 1438155

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13554

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Fri Aug  3 11:14:01 CEST 2018 on sn-devel-144

- - - - -
a3d248f2 by Oleksandr Natalenko at 2018-08-03T15:28:52Z
systemd: Only start smb when network interfaces are up

For smb, if the smb.conf contains explicit bindings to the network
interfaces, the service must wait till network interfaces are up,
otherwise the service won't be operational.

The 0e571054a61e commit and the BZ 13184 have fixed this for nmb and
samba, so do exactly the same here, for smb.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13559

Signed-off-by: Oleksandr Natalenko <oleksandr at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Aug  3 17:28:52 CEST 2018 on sn-devel-144

- - - - -
97702ffc by Justin Stephenson at 2018-08-06T00:46:16Z
Add net lookup options

Add missing net lookup options to net man page

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jim McDonough <jmcd at samba.org>

- - - - -
dea788e5 by Gary Lockyer at 2018-08-06T03:36:42Z
dns scavenging: Add extra tests for custom filter

Add extra tests for the custom ldb filter used by the dns scavenging
code.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Aug  6 05:36:43 CEST 2018 on sn-devel-144

- - - - -
8f83933f by Aaron Haslett at 2018-08-06T03:37:42Z
tdb: adding readonly locks mode to tdbbackup tool

The netcmd 'domain backup offline' command will use the tdbbackup tool but
require readonly locking of tdb databases, otherwise all database access would
be blocked during a backup.  This patch adds the option.  A backup script
should use this tool with the readonly locks option after taking a transaction
lock on the target database.

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
04217372 by Aaron Haslett at 2018-08-06T03:37:42Z
tdb: test for readonly locks mode on tdbbackup command

Simple bash test for readonly locks on tdbbackup:
1. Running tdbbackup on a database with and without readonly locks enabled.
2. Dump both backups and original.
3. Check all three dumps match.

A binary sample_tdb.tdb file is included for the test because the existing
sample tdbs in lib/tdb/test are either corrupt or empty.

Signed-off-by: Aaron Haslett <aaron.haslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4f532cc1 by Tim Beale at 2018-08-06T03:37:42Z
netcmd: Improve domain backup targetdir checks

+ Added check that specified targetdir is actually a directory (if it
exists)
+ Deleted a redundant 'Creating targetdir' check that would never be hit
+ Move code into a separate function so we can reuse it for offline
backups (which take a different set of parameters, but still have a
targetdir)

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
f17d2012 by Aaron Haslett at 2018-08-06T03:37:42Z
netcmd: domain backup offline command

Unlike the existing 'domain backup online' command, this command allows an
admin to back up a local samba installation using the filesystem and the
tdbbackup tool instead of using remote protocols.  It replaces samba_backup
as that tool does not handle sam.ldb and secrets.ldb correctly.  Those two
databases need to have transactions started on them before their downstream
ldb and tdb files are backed up.

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
2104818e by Tim Beale at 2018-08-06T03:37:43Z
tests: New offline backup tests with tweaks to old online classes

Offline backups have a slightly different syntax, as they don't take the
server or user-creds parameters. In the untar case, the offline backup
will actually have the secrets present, so making asserting on this
more flexible.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
2800611d by Aaron Haslett at 2018-08-06T03:37:43Z
netcmd: domain backup offline command - offline test with ldapcmp

This test checks that when you do an offline backup and restore or untar it,
the restored database is the same as the original.  Test is repeated for
'mdb' and 'tdb' database backends.

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
eb4161d7 by Aaron Haslett at 2018-08-06T06:45:19Z
selftest: offline backup restore target

This is a selftest target built from a restored offline backup.
Other backup routines are modified to remove the assumption that every backup
requires server and credentials arguments, since offline backup doesn't
want them.  Also, prepare_dc_testenv now returns the generated ctx so we can
run or re-run routines that require it later.

Signed-off-by: Aaron Haslett <aaron.haslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Mon Aug  6 08:45:19 CEST 2018 on sn-devel-144

- - - - -
d76c7b20 by Richard Sharpe at 2018-08-06T06:46:41Z
Minor, really small, documentation fix.

Signed-off-by: Richard Sharpe <realrichardsharpe at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c6b95c36 by Swen Schillig at 2018-08-06T09:37:32Z
ctdb: remove queue destructor as it isn't needed anymore

After

commit e097b7f8ff1a9992de1d11474dac4969e30cd679
Author: David Disseldorp <ddiss at suse.de>
Date:   Sun Jul 31 03:14:54 2011 +0200

    io: Make queue_io_read() safe for reentry

the destructor has no purpose anymore, therfore, remove it.

Signed-off-by: Swen Schillig <swen at vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: David Disseldorp <ddiss at samba.org>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Mon Aug  6 11:37:32 CEST 2018 on sn-devel-144

- - - - -
0530cccc by Andreas Schneider at 2018-08-06T23:49:34Z
s3:waf: Install eventlogadm to /usr/sbin

The eventlogadm binary needs write access to the registry which, by
default, is only possible as root.

https://bugzilla.samba.org/show_bug.cgi?id=13561

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Aug  7 01:49:34 CEST 2018 on sn-devel-144

- - - - -
5b54ced3 by Volker Lendecke at 2018-08-08T18:22:05Z
smbd: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
60c0a2b3 by Volker Lendecke at 2018-08-08T18:22:05Z
smbd: Fix CID 1438246 Unchecked return value

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
01d9be27 by Volker Lendecke at 2018-08-08T18:22:05Z
smbd: Fix CID 1438245 Dereference before null check

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ee3dd90a by Volker Lendecke at 2018-08-08T18:22:05Z
libsmb: Fix CID 1438244 Unsigned compared against 0

ndr_size_dom_sid returns a size_t, so that can't be <0. Also, the only
case that ndr_size_dom_sid returns 0 is a NULL sid
pointer. ndr_size_dom_sid can reasonably be assumed to not overflow, the
number of sub-auths is a uint8. That times 4 plus 8 always fits into a
size_t.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0c8174cf by Volker Lendecke at 2018-08-08T21:10:22Z
libsmb: Fix CID 1438243 Unchecked return value

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Aug  8 23:10:22 CEST 2018 on sn-devel-144

- - - - -
bf9cb64d by Anoop C S at 2018-08-08T23:16:22Z
s3/locking: Corrections and improvements to inline comments

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Jim McDonough <jmcd at samba.org>

- - - - -
9b105651 by Justin Stephenson at 2018-08-09T02:06:17Z
Shorten description in vfs_linux_xfs_sgid manual

this fixes a lexgrog parse error, the NAME subheader description
of the vfs_linux_xfs_sgid(8) manual was too long, this will shorten
the description and allow it to be correctly detected by mandb.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13562

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Aug  9 04:06:17 CEST 2018 on sn-devel-144

- - - - -
bd64af6b by David Disseldorp at 2018-08-09T11:29:15Z
ctdb/build: link ctdb_mutex_ceph_rados_helper against ceph-common

ceph-common linkage is needed with new versions of Ceph.
Also respect the --libcephfs_dir=<path> parameter when provided.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
85706bd2 by Samuel Cabrero at 2018-08-09T11:29:15Z
ctdb_mutex_ceph_rados_helper: Set SIGINT signal handler

Set a handler for SIGINT to release the lock.

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8d30fd59 by David Disseldorp at 2018-08-09T11:29:15Z
ctdb_mutex_ceph_rados_helper: use talloc destructor for cleanup

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
91a89c14 by David Disseldorp at 2018-08-09T11:29:15Z
ctdb_mutex_ceph_rados_helper: rename timer_ev to ppid_timer_ev

In preparation for adding a lock refresh timer.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
ce289e89 by David Disseldorp at 2018-08-09T11:29:15Z
ctdb_mutex_ceph_rados_helper: fix deadlock via lock renewals

RADOS locks without expiry persist indefinitely. This results in CTDB
deadlock during failover if the recovery master dies unexpectedly, as
subsequently elected recovery master nodes can't obtain the recovery
lock.
Avoid deadlock by using a lock expiration time (10s by default), and
renewing it periodically.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13540

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4abf348e by David Disseldorp at 2018-08-09T14:26:36Z
ctdb: add expiry test for ctdb_mutex_ceph_rados_helper

Kill the ctdb_mutex_ceph_rados_helper with SIGKILL and then confirm
that the lock is automatically released following expiry.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): David Disseldorp <ddiss at samba.org>
Autobuild-Date(master): Thu Aug  9 16:26:36 CEST 2018 on sn-devel-144

- - - - -
8479401b by Andreas Schneider at 2018-08-09T17:57:02Z
lib: Add support to parse MS Catalog files

Signed-off-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Thu Aug  9 19:57:02 CEST 2018 on sn-devel-144

- - - - -
9c131254 by Noel Power at 2018-08-10T00:43:33Z
s3/smbd: Ensure quota code is only called when quota support detected

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13563

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Aug 10 02:43:33 CEST 2018 on sn-devel-144

- - - - -
94ffd4b7 by Amitay Isaacs at 2018-08-10T00:44:36Z
dlz-bind: Add support for BIND 9.12.x

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1a86733d by Amitay Isaacs at 2018-08-10T03:36:19Z
provision: Add support for BIND 9.12.x

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Aug 10 05:36:19 CEST 2018 on sn-devel-144

- - - - -
75287495 by Douglas Bagnall at 2018-08-10T07:27:03Z
samba-tool drs showrepl tests: improve debugging for mystery error

Under some circumstances the samba-tool command is failing with no
stdout output at all, leaving few clues in the logs.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Aug 10 09:27:03 CEST 2018 on sn-devel-144

- - - - -
6b68e3ec by Anoop C S at 2018-08-10T16:05:09Z
s3/libsmb: Explicitly set delete_on_close token for rmdir

The current implementation of `rmdir` hopes to get the directory deleted
on closing last open handle when FILE_DELETE_ON_CLOSE is set on it. But
for non-empty directories Windows doesn't error out during an open call.
Following that we internally refuse to set initial delete_on_close while
opening a non-empty directory. This prevents us from trying to delete
the directory when last open handle is closed.

Instead of relying on FILE_DELETE_ON_CLOSE during an open we explicitly
set delete_on_close token on directory handle once it is available. This
ensures that NT_STATUS_DIRECTORY_NOT_EMPTY is returned for `rmdir` on
non-empty directories while closing open directory handle.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13204

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
6a7f1174 by Anoop C S at 2018-08-10T16:05:09Z
s4/torture: Add new test for DELETE_ON_CLOSE on non-empty directories

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13204

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
bca40084 by Jeremy Allison at 2018-08-10T19:08:14Z
s3: tests: smbclient. Regression test to ensure we get NT_STATUS_DIRECTORY_NOT_EMPTY on rmdir.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13204

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Aug 10 21:08:14 CEST 2018 on sn-devel-144

- - - - -
e6689c3e by Andreas Schneider at 2018-08-10T23:49:16Z
wbinfo: Free memory when we leave wbinfo_dsgetdcname()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e4f4f5eb by Andreas Schneider at 2018-08-10T23:49:16Z
s3:passdb: Don't leak memory on error in fetch_ldap_pw()

Found by covscan.

A candidate to use tallac ...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f20150fb by Andreas Schneider at 2018-08-10T23:49:16Z
s3:utils: Do not overflow the destination buffer in net_idmap_restore()

Found by covsan.

error[invalidScanfFormatWidth]: Width 128 given in format string (no. 2)
is larger than destination buffer 'sid_string[128]', use %127s to
prevent overflowing it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b7b4fc51 by Andreas Schneider at 2018-08-10T23:49:16Z
s3:utils: Do not leak memory in new_user()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d4fb124a by Andreas Schneider at 2018-08-10T23:49:16Z
s4:lib: Fix a possible fd leak in gp_get_file()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3d32c026 by Andreas Schneider at 2018-08-10T23:49:16Z
s3:client: Avoid a possible fd leak in do_get()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dbdbd487 by Andreas Schneider at 2018-08-10T23:49:16Z
s3:libads: Fix memory leaks in ads_krb5_chg_password()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3e6ce5c6 by Andreas Schneider at 2018-08-11T02:43:15Z
s3:registry: Fix possible memory leak in _reg_perfcount_multi_sz_from_tdb()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Aug 11 04:43:15 CEST 2018 on sn-devel-144

- - - - -
4c0b49b3 by Andreas Schneider at 2018-08-13T17:46:08Z
s3:winbind: Fix memory leak in nss_init()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4909b966 by Ralph Wuerthner at 2018-08-13T17:46:08Z
s3: vfs: time_audit: fix handling of token_blob in smb_time_audit_offload_read_recv()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13568

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a98f09a0 by Christof Schmitt at 2018-08-13T20:35:20Z
selftest: Load time_audit and full_audit modules for all tests

Previously the only test was to load these modules to trigger the
smb_vfs_assert_all_fns check. As these modules just pass through the
calls, they can be loaded for all tests to ensure that the codepaths are
exercised. This would have found the problem in
smb_time_audit_offload_read_recv.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13568

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Mon Aug 13 22:35:20 CEST 2018 on sn-devel-144

- - - - -
6b3cc791 by Volker Lendecke at 2018-08-14T06:54:17Z
g_lock: Avoid a double call to serverid_exist

If we try to G_LOCK_READ while a G_LOCK_WRITE is active, we do the
serverid_exists call twice. Avoid that.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
00513daf by Volker Lendecke at 2018-08-14T09:42:10Z
g_lock: Simplify g_lock_trylock

While chasing a bug in g_lock (not in master) I saw some opportunity to
simplify g_lock_trylock a bit. This is array handling, and array
handling is just extremely error-prone. This *might* be a little less
efficient or large numbers of READ locks, but this remains to be
seen. For now, simplify the code.

First, we make two passes now: One to remove ourselves, and the other
one to search for conflicts. Mixing up both made it pretty hard for me
to follow the code.

Second, I've removed the _mylock and mylock pointer/struct logic and
replaced it with the "mylock.pid.pid != 0 ? &mylock : NULL" when calling
g_lock_store. To me, this focuses the logic whether to add ourselves in
one place instead of spreading it around in the whole routine.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Aug 14 11:42:10 CEST 2018 on sn-devel-144

- - - - -
5edcaece by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1139 libcli/auth: Add initial tests for ntlm_password_check()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
1d89fe91 by Günther Deschner at 2018-08-14T11:57:15Z
CVE-2018-1139 libcli/auth: fix debug messages in hash_password_check()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

CVE-2018-1139: Weak authentication protocol allowed.

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
947cf385 by Günther Deschner at 2018-08-14T11:57:15Z
CVE-2018-1139 s3-utils: use enum ntlm_auth_level in ntlm_password_check().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

CVE-2018-1139: Weak authentication protocol allowed.

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c88f7795 by Günther Deschner at 2018-08-14T11:57:15Z
CVE-2018-1139 selftest: verify whether ntlmv1 can be used via SMB1 when it is disabled.

Right now, this test will succeed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

CVE-2018-1139: Weak authentication protocol allowed.

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c25460ee by Günther Deschner at 2018-08-14T11:57:15Z
CVE-2018-1139 libcli/auth: Do not allow ntlmv1 over SMB1 when it is disabled via "ntlm auth".

This fixes a regression that came in via 00db3aba6cf9ebaafdf39ee2f9c7ba5ec2281ea0.

Found by Vivek Das <vdas at redhat.com> (Red Hat QE).

In order to demonstrate simply run:

smbclient //server/share -U user%password -mNT1 -c quit \
--option="client ntlmv2 auth"=no \
--option="client use spnego"=no

against a server that uses "ntlm auth = ntlmv2-only" (our default
setting).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

CVE-2018-1139: Weak authentication protocol allowed.

Guenther

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
0998f2f1 by Andrej Gessel at 2018-08-14T11:57:15Z
CVE-2018-1140 Add NULL check for ldb_dn_get_casefold() in ltdb_index_dn_attr()

Signed-off-by: Andrej Gessel <Andrej.Gessel at janztec.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
3f95957d by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1140 ldb: Check for ldb_dn_get_casefold() failure in ldb_sqlite

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
b27d9733 by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1140 ldb_tdb: Ensure the dn in distinguishedName= is valid before use

ldb_dn_from_ldb_val() does not validate this untrusted input, so a later
call to ldb_dn_get_casefold() can fail if the input is not valid.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
3c1fbb18 by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1140 ldb_tdb: Check for DN validity in add, rename and search

This ensures we fail with a good error code before an eventual ldb_dn_get_casefold() which
would otherwise fail.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
b6b72d00 by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1140 ldb: Add tests for search add and rename with a bad dn= DN

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
b7f0ee93 by Andrew Bartlett at 2018-08-14T11:57:15Z
Release LDB 1.5.0 for CVE-2018-1140

* Security fix for CVE-2018-1140 (NULL pointer de-reference, bug 13374)
* Fix memory leaks and missing error checks (bug 13459, 13471, 13475)

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
be4c0938 by Kai Blin at 2018-08-14T11:57:15Z
CVE-2018-1140 dns: Add a test to trigger the LDB casefolding issue on invalid chars

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466

Signed-off-by: Kai Blin <kai at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
21d628e0 by Tim Beale at 2018-08-14T11:57:15Z
CVE-2018-10919 security: Move object-specific access checks into separate function

Object-specific access checks refer to a specific section of the
MS-ADTS, and the code closely matches the spec. We need to extend this
logic to properly handle the Control-Access Right (CR), so it makes
sense to split the logic out into its own function.

This patch just moves the code, and should not alter the logic (apart
from ading in the boolean grant_access return variable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c107e2d6 by Tim Beale at 2018-08-14T11:57:15Z
CVE-2018-10919 security: Add more comments to the object-specific access checks

Reading the spec and then reading the code makes sense, but we could
comment the code more so it makes sense on its own.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
375f48f7 by Tim Beale at 2018-08-14T11:57:15Z
CVE-2018-10919 tests: Add tests for guessing confidential attributes

Adds tests that assert that a confidential attribute cannot be guessed
by an unprivileged user through wildcard DB searches.

The tests basically consist of a set of DB searches/assertions that
get run for:
- basic searches against a confidential attribute
- confidential attributes that get overridden by giving access to the
  user via an ACE (run against a variety of ACEs)
- protecting a non-confidential attribute via an ACL that denies read-
  access (run against a variety of ACEs)
- querying confidential attributes via the dirsync controls

These tests all pass when run against a Windows Dc and all fail against
a Samba DC.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
9eb8340e by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 tests: Add test case for object visibility with limited rights

Currently Samba is a bit disclosive with LDB_OP_PRESENT (i.e.
attribute=*) searches compared to Windows.

All the acl.py tests are based on objectClass=* searches, where Windows
will happily tell a user about objects they have List Contents rights,
but not Read Property rights for. However, if you change the attribute
being searched for, suddenly the objects are no longer visible on
Windows (whereas they are on Samba).

This is a problem, because Samba can tell you about which objects have
confidential attributes, which in itself could be disclosive.

This patch adds a acl.py test-case that highlights this behaviour. The
test passes against Windows but fails against Samba.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
563e454e by Gary Lockyer at 2018-08-14T11:57:16Z
CVE-2018-10919 tests: test ldap searches for non-existent attributes.

It is perfectly legal to search LDAP for an attribute that is not part
of the schema.  That part of the query should simply not match.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ba46578f by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 security: Fix checking of object-specific CONTROL_ACCESS rights

An 'Object Access Allowed' ACE that assigned 'Control Access' (CR)
rights to a specific attribute would not actually grant access.

What was happening was the remaining_access mask for the object_tree
nodes would be Read Property (RP) + Control Access (CR). The ACE mapped
to the schemaIDGUID for a given attribute, which would end up being a
child node in the tree. So the CR bit was cleared for a child node, but
not the rest of the tree. We would then check the user had the RP access
right, which it did. However, the RP right was cleared for another node
in the tree, which still had the CR bit set in its remaining_access
bitmap, so Samba would not grant access.

Generally, the remaining_access only ever has one bit set, which means
this isn't a problem normally. However, in the Control Access case there
are 2 separate bits being checked, i.e. RP + CR.

One option to fix this problem would be to clear the remaining_access
for the tree instead of just the node. However, the Windows spec is
actually pretty clear on this: if the ACE has a CR right present, then
you can stop any further access checks.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
80c4e17f by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 acl_read: Split access_mask logic out into helper function

So we can re-use the same logic laster for checking the search-ops.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4234579a by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 acl_read: Small refactor to aclread_callback()

Flip the dirsync check (to avoid a double negative), and use a helper
boolean variable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
98c2e6a1 by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 acl_read: Flip the logic in the dirsync check

This better reflects the special case we're making for dirsync, and gets
rid of a 'if-else' clause.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
fc45da52 by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 acl_read: Fix unauthorized attribute access via searches

A user that doesn't have access to view an attribute can still guess the
attribute's value via repeated LDAP searches. This affects confidential
attributes, as well as ACLs applied to an object/attribute to deny
access.

Currently the code will hide objects if the attribute filter contains an
attribute they are not authorized to see. However, the code still
returns objects as results if confidential attribute is in the search
expression itself, but not in the attribute filter.

To fix this problem we have to check the access rights on the attributes
in the search-tree, as well as the attributes returned in the message.

Points of note:
- I've preserved the existing dirsync logic (the dirsync module code
  suppresses the result as long as the replPropertyMetaData attribute is
  removed). However, there doesn't appear to be any test that highlights
  that this functionality is required for dirsync.
- To avoid this fix breaking the acl.py tests, we need to still permit
  searches like 'objectClass=*', even though we don't have Read Property
  access rights for the objectClass attribute. The logic that Windows
  uses does not appear to be clearly documented, so I've made a best
  guess that seems to mirror Windows behaviour.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
7070aa38 by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 tests: Add extra test for dirsync deleted object corner-case

The acl_read.c code contains a special case to allow dirsync to
work-around having insufficient access rights. We had a concern that
the dirsync module could leak sensitive information for deleted objects.
This patch adds a test-case to prove whether or not this is happening.

The new test case is similar to the existing dirsync test except:
- We make the confidential attribute also preserve-on-delete, so it
  hangs around for deleted objcts. Because the attributes now persist
  across test case runs, I've used a different attribute to normal.
  (Technically, the dirsync search expressions are now specific enough
  that the regular attribute could be used, but it would make things
  quite fragile if someone tried to add a new test case).
- To handle searching for deleted objects, the search expressions are
  now more complicated. Currently dirsync adds an extra-filter to the
  '!' searches to exclude deleted objects, i.e. samaccountname matches
  the test-objects AND the object is not deleted. We now extend this to
  include deleted objects with lastKnownParent equal to the test OU.
  The search expression matches either case so that we can use the same
  expression throughout the test (regardless of whether the object is
  deleted yet or not).

This test proves that the dirsync corner-case does not actually leak
sensitive information on Samba. This is due to a bug in the dirsync
code - when the buggy line is removed, this new test promptly fails.
Test also passes against Windows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
860f575f by Jeremy Allison at 2018-08-14T11:57:16Z
libsmb: Ensure smbc_urlencode() can't overwrite passed in buffer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13453

CVE-2018-10858: Insufficient input validation on client directory
		listing in libsmbclient.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
be3d4b2d by Jeremy Allison at 2018-08-14T11:57:16Z
libsmb: Harden smbc_readdir_internal() against returns from malicious servers.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13453

CVE-2018-10858: Insufficient input validation on client directory
		listing in libsmbclient.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
a45de51c by Andrew Bartlett at 2018-08-14T15:02:38Z
cracknames: Fix DoS (NULL pointer de-ref) when not servicePrincipalName is set on a user

This regression was introduced in Samba 4.7 by bug 12842 and in
master git commit eb2e77970e41c1cb62c041877565e939c78ff52d.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13552

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(master): Tue Aug 14 17:02:38 CEST 2018 on sn-devel-144

- - - - -
e4f38b06 by Timur I. Bakeyev at 2018-08-14T17:08:25Z
ldb tests: fix assertion on wrong pointer

We are allocating msg02, but check in assertion msg01, which makes no
sense here.

Signed-off-by: Timur I. Bakeyev <timur at freebsd.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9eccf6a1 by Andreas Schneider at 2018-08-14T20:02:06Z
s3:libads: Free addr before we free the context

Introduced by dbdbd4875ecac3e7334750f46f1f494b7afe6628

CID 1438395

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Aug 14 22:02:06 CEST 2018 on sn-devel-144

- - - - -
b9e2a2de by Joe Guo at 2018-08-15T05:08:23Z
ldb: no need to call del_transaction in ldb_transaction_commit

No matter commit succeeded or failed, transation will be delete afterwards.
So there is no need to delete it here.

Aganst Samba this causes an `LDAP error 51 LDAP_BUSY` error when the transaction
fails, say while we try to add users to groups in large amount and
the original error is lost.

In Samba, the rootdse module fails early in the del part of the
start/end/del pattern, and in ldb_tdb and ldb_mdb a failed commit
always ends the transaction, even on failure.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d645546a by Andrej Gessel at 2018-08-15T05:08:24Z
fix mem leak in ltdb_index_dn_base_dn and ltdb_search_indexed

Signed-off-by: Andrej Gessel <Andrej.Gessel at janztec.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d71c655e by Andrej Gessel at 2018-08-15T05:08:24Z
fix mem leak in ldbsearch

Signed-off-by: Andrej Gessel <Andrej.Gessel at janztec.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
542e7c17 by Andrew Bartlett at 2018-08-15T05:08:24Z
ldb_tdb: Remove pointless check of ldb_dn_is_valid()

If the DN is not valid the ltdb_search_dn1() will catch it with ldb_dn_validate() which
is the only safe way to check this.  ldb_dn_is_valid() does not actually check, but instead
returns only the result of the previous checks, if there was one.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
2dafbd32 by Andrew Bartlett at 2018-08-15T05:08:24Z
ldb: Add new function ldb_dn_add_child_val()

This is safer for untrusted input than ldb_dn_add_child_fmt()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
9d46795b by Andrew Bartlett at 2018-08-15T05:08:24Z
ldb: extend API tests

These additional API tests just check that an invalid base DN
is never accepted.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
bdbb9422 by Andrew Bartlett at 2018-08-15T05:08:24Z
ldb: Release LDB 1.5.1

* New API ldb_dn_add_child_val() avoids passing untrusted input to
  ldb_dn_add_child_fmt() (bug 13466)
* Free memory nearer to the allocation in calls made by ldbsearch
* Do not overwrite ldb_transaction_commit failure error messages
  with a pointless del_transaction()

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
aa01203f by Andrew Bartlett at 2018-08-15T05:08:24Z
dns_server: Be strict when constructing a LDB DN from an untrusted DNS name

This changes our DNS server to be much more careful when constructing DNS names
into LDB DN values.

This avoids a segfault deep in the LDB code if the ldb_dn_get_casefold() fails there.

A seperate patch will address that part of the issue, and a later patch
will re-work this code to use single API: ldb_dn_add_child_val().  This
is not squahed with this work because this patch does not rely on a new
LDB release, and so may be helpful for a backport.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
28e2a518 by Andrew Bartlett at 2018-08-15T05:08:24Z
dns_server: Avoid ldb_dn_add_child_fmt() on untrusted input

By using the new ldb_dn_add_child_val() we ensure that the user-controlled values are
not parsed as DN seperators.

Additionally, the casefold DN is obtained before the search to trigger
a full parse of the DN before being handled to the LDB search.

This is not normally required but is done here due to the nature
of the untrusted input.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
badd7a23 by Joe Guo at 2018-08-15T05:08:25Z
samba-tool/drs: set dns_backend to SAMBA_INTERNAL in cmd_drs_clone_dc_database

The default value is "NONE", need to specify it to use SAMBA_INTERNAL so
that the DNS partitions are replicated.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
8084f183 by Joe Guo at 2018-08-15T05:08:25Z
emulate/traffic: fix next usage

In commit b0c9de820c07d77c03b80505cb811ac1dac0808f, line 343:

    self.next_conversation_id = itertools.count().next

was changed to:

    self.next_conversation_id = next(itertools.count())

which is not correct, the first one is a function, the second one is a
int. This patch fixed it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13573

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
68c64c63 by Joe Guo at 2018-08-15T05:08:25Z
traffic: uniform stats output

The original code is trying to output different data format for tty or file.
This is unnecessary and cause confusion while writing script to parse result.

The human-readable one is also easy for code to parse.
Remove if check for isatty(), just make output the same.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
ceed07fe by Joe Guo at 2018-08-15T05:08:25Z
traffic-replay: add extra check

Make sure --average-groups-per-user is not more than --number-of-users

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
371c5c70 by Joe Guo at 2018-08-15T05:08:25Z
emulate/traffic: add sAMAccountName in create_group

While using script/traffic_replay to generate users and groups, we get
autogenerated group name like:

    $2A6F42B2-39FAF4556E2BE379

This patch specify sAMAccountName to overwriten the name.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1da4ff2e by Timur I. Bakeyev at 2018-08-15T05:08:25Z
third_party:build: Test for the flags, recognized by Clang.

Make amd64 SYSTEM_UNAME_MACHINE an alias for x86_64.

Signed-off-by: Timur I. Bakeyev <timur at iXsystems.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
14077b66 by Tim Beale at 2018-08-15T05:08:25Z
netcmd: domain backup didn't support prompting for password

The online/rename backups only worked if you specified both the username
and password in the actual command itself. If you just entered the
username (expecting to be prompted for the password later), then the
command was rejected.

The problem was the order the code was doing things in. We were checking
credopts.creds.get_password() *before* we'd called
credopts.get_credentials(lp), whereas it should be the other way
around.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13566

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
8fb706c3 by Tim Beale at 2018-08-15T05:08:25Z
netcmd: Fix kerberos option for domain backups

The previous fix still didn't work if you specified --kerberos=yes (in
which case the creds still doesn't have a password).

credopts.get_credentials(lp) should be enough to ensure a user/password
is set (it's all that the other commands seem to do).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13566

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d2d03951 by Tim Beale at 2018-08-15T05:08:26Z
netcmd: Delete unnecessary function

Minor code cleanup. The last 2 patches gutted this function, to the
point where there's no longer any value in keeping it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13566

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
f249bea1 by Tim Beale at 2018-08-15T08:19:09Z
netcmd: Fix --kerberos=yes and --no-secrets domain backups

The --kerberos=yes and --no-secrets options didn't work in combination
for domain backups. The problem was creds.get_username() might not
necessarily match the kerberos user (such as in the selftest
environment). If this was the case, then trying to reset the admin
password failed (because the creds.get_username() didn't exist in
the DB).

Because the admin user always has a fixed RID, we can work out the
administrator based on its object SID, instead of relying on the
username in the creds.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13566

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Aug 15 10:19:09 CEST 2018 on sn-devel-144

- - - - -
739691fd by Andrew Bartlett at 2018-08-16T21:42:19Z
buildtools: Split git ls-files output on newline, not any whitespace

This allows files to have a space in the filename within the Samba git tree.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
6d52ef8d by Garming Sam at 2018-08-16T21:42:19Z
git: Treat .dump files as binary

This means that git grep will no longer show TDB dumps. This can be
changed at runtime using -a for all to include these files, while -I
will also omit any references to the files (no Binary file * matches).

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ea297d0c by Garming Sam at 2018-08-16T21:42:19Z
winreg: Add hyper REG_QWORD to parsing routines

This will be useful when exporting registry.pol files.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
572fd631 by Garming Sam at 2018-08-16T21:42:19Z
preg: Build python preg bindings

These will be used in the GPO import/export.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
463dcc90 by Garming Sam at 2018-08-16T21:42:19Z
preg: Unpack winreg_Data for parsing

It seems that there might be pre-existing endianness issues which would be fixed by the ndr_push.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c71ba94c by Garming Sam at 2018-08-16T21:42:20Z
preg: Using winreg_Data_GPO instead of DATA_BLOB

We need to make a duplicate in order to have reasonable python bindings.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8140a7bb by Garming Sam at 2018-08-16T21:42:20Z
preg: Use gensize to allow modification of winreg data to be repacked

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7047f457 by Garming Sam at 2018-08-16T21:42:20Z
gp_parse: Introduce new module for parsing GPO files

This is the default parser which will cause the file to be restored
as-is -- leaving only an effectively blank XML file as a placeholder.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
34453a08 by Garming Sam at 2018-08-16T21:42:20Z
gpo: Add a backup command (similar to fetch)

The idea behind this command is that you will eventually backup a number
of XML files which can be user-editable and have generic entities to be
later restored in the same domain or a different domain.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
aac6cd37 by Garming Sam at 2018-08-16T21:42:21Z
gpo: Add a restore command (for backups) from XML

Currently because no parsers have been written, this just copies the old
files and puts them in their places.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6c5a5077 by Garming Sam at 2018-08-16T21:42:21Z
gp_pol: Parse the .pol files (PReg) which stored winreg settings

Currently, we do not look inside the .pol files for any settings (and do
not generalize any so far).

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e27c8689 by Garming Sam at 2018-08-16T21:42:21Z
gp_ini: Parse .ini files in SYSVOL

These are fdeploy, scripts + psscripts as well as the GPT.ini at the top
level. Note that GPT.ini has a different character encoding and we
specify it here.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4a69308b by Garming Sam at 2018-08-16T21:42:21Z
gp_csv: Parse the audit.csv file which records audit settings

Based on the setting, the csv will omit certain fields. Using this we
can later infer as to how to generalize the ACLs and SIDs.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dac3c204 by Garming Sam at 2018-08-16T21:42:21Z
gp_inf: Parse the GptTmpl.inf file which stores security settings

This is NOT an ini file and CANNOT be parsed by Python ConfigParser
without losing information (it would likely eat meaningful whitespace
and so should not be done).

There are three main types of settings:

 * Name,Mode,ACL
 * key = value
 * registry key and value

   Note: This appears as key=value, but registry keys in the general
   case may have = in their names, so we record the entire string in
   order to be as safe as possible.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
497db985 by Garming Sam at 2018-08-16T21:42:21Z
gp_aas: Leave a placeholder for the .aas files for now

This is to be implemented, but the documentation is somewhat lacking for
the .aas files and we so we leave this for now. In particular, the
documentation doesn't seem to describe all the possible sections, nor do
we understand what happens if we replace certain aspects of the file --
and whether or not it will remain functional.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
854c3eb2 by Garming Sam at 2018-08-16T21:42:21Z
gpo: Enable more specific parsers of GPO files

* .pol files
* .ini (and GPT.ini)
* audit.csv
* GptTmpl.inf

.aas is currently not handled.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ee010392 by Garming Sam at 2018-08-16T21:42:21Z
gp_csv: Add CSV generalization metadata

There are user identifiers and ACLs which may be stored in the audit CSV.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4edb1769 by Garming Sam at 2018-08-16T21:42:21Z
gp_ini: Allow better overriding of behaviour in inherited classes

We will need this to parse the parameters or section names as SIDs for fdeploy1.ini

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
806e3e60 by Garming Sam at 2018-08-16T21:42:21Z
gp_ini: Add a fdeploy1 parser for better generalization

We still fail to handle entities in fdeploy.ini (version 0) files. Here we
manage to factor out some of the SIDs, but not all of them. This will be
completed in a later patch. The overall idea is to split the SID values into
individual XML elements and annotate them. We also note down network paths for
the redirection folders.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
57dd88ce by Garming Sam at 2018-08-16T21:42:22Z
gp_ini: Add a scripts ini parser for better generalization

We mark the command path argument as a network path.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
abff0c4f by Garming Sam at 2018-08-16T21:42:22Z
gp_parse: Add a generalize XML function to the top level parser

In this function we take XML and using the required metadata, we rewrite
it into a generic form using entities. ElementTree unfortunately does
not allow us to store unescaped entities, and so we must do a textual
replace on the output XML.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3ff695fd by Garming Sam at 2018-08-16T21:42:22Z
fdeploy_ini: Generalize the share name SIDs

This overrides the custom entity handler defined in the top level parser.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
07de1565 by Garming Sam at 2018-08-16T21:42:22Z
gpo: Add a --generalize to the backup command

This normally prints out the entities in DTD form to be given to the restore
command with --entities. Specifying --entities during the backup conveniently
writes these entities to a file. Generalizing occurs after the standard backup
on the XML files, which will then re-write the XML file.

There are a number of files which can be further handled, including many of the
preferences XML files. This will require more annotation and parsing.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9977b4bc by Garming Sam at 2018-08-16T21:42:22Z
gitattributes: Ignore .SAMBABACKUP files

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cedfea1b by Garming Sam at 2018-08-16T21:42:22Z
gpo: Make restore with entities more robust

Sometimes the restore fails for unknown reasons, but rearranging the XML
such that the DTD is after the xml header appears to fix it. This might
be the case in certain files where no entities are used perhaps.

This could probably be made more tolerant using regex, but for the most
part we expect the fixed output from the minidom pretty-printed XML.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a844137e by Garming Sam at 2018-08-16T21:42:22Z
tests/gpo: Add a backup for showing that GPO backup and restore works

This will be used to write a test in the coming patches.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
53ee5157 by Garming Sam at 2018-08-16T21:42:22Z
tests/gpo: Tests using a static backup directory from gpo backup

In particular, we want to see that the binary matches, that the XML will
backup to the same values, that the fallback with copy-restore works,
and that the generalize will generalize over different restored
entities.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f4f0a5d9 by Garming Sam at 2018-08-16T21:42:22Z
gpo: Maintain an XML DTD for reference of the backup

This may or may not actually parse, but is mostly for reference

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cdff99b9 by Andrew Bartlett at 2018-08-16T21:42:22Z
gpo: Always use an SMB signed connection

This ensures data integrity in the backup.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
3990df08 by Andrew Bartlett at 2018-08-17T00:44:41Z
WHATSNEW: Add information on new GPO features

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Aug 17 02:44:41 CEST 2018 on sn-devel-144

- - - - -
d313e0e4 by Joe Guo at 2018-08-17T00:58:26Z
descriptor: add missing backslash for long sddl str

Find this bug while doing PEP8.
We are lucky this code was not used yet.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
2f371493 by Joe Guo at 2018-08-17T00:58:26Z
python/samba/tests: fix SamDB dummy replacement

In commit 6de9d878b, a dummy SamDB lambda was added:

    SamDB = lambda *x: None

The `*x` will only cover positional args. If we call it with kwargs:

    samdb = SamDB(url=url)

We will get TypeError:

    <lambda>() got an unexpected keyword argument 'url'

This commit fix this. It also fix PEP8 E731:

    do not assign a lambda expression, use a def

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
cd3b06fa by Joe Guo at 2018-08-17T00:58:26Z
python3: reuse cmp_fn defined in compat.py

This will also fix PEP8 E306:

    expected 1 blank line before a nested definition, found 0

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a39c8f44 by Tim Beale at 2018-08-17T00:58:26Z
Fix PEP8 warning E711 comparison to None

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
96b726ea by Tim Beale at 2018-08-17T00:58:26Z
Fix PEP8 warning E201/202/203 array/dict whitespace

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
9b86c5f8 by Tim Beale at 2018-08-17T00:58:26Z
Fix PEP8 warning E122/E126/E127 wrong indent for continuation lines

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
078bd795 by Tim Beale at 2018-08-17T00:58:26Z
python/pso tests: use  string .format() style rather than C-style %s/%d.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
3a050542 by Andrew Bartlett at 2018-08-17T00:58:27Z
samba-tool domain passwordsettings: Avoid except Exception

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
30e6e04c by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning F841 local variable 'blah' is assigned to but never used

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1a30a68b by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning E225 missing whitespace around operator

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e741a193 by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning F401 'blah' imported but unused

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
092130be by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning E231 missing whitespace after ','

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
b8920aab by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning E302 expected 2 blank lines

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
87b919b6 by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning E303 too many blank lines

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
26ece82e by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning W291 trailing whitespace

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
2bd0a208 by Tim Beale at 2018-08-17T00:58:28Z
Fix PEP8 warning W503 line break before binary operator

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a86e2b34 by Tim Beale at 2018-08-17T00:58:28Z
Refactor for PEP8 warning E501 line too long

The attribute names here are so long it means there's not a very nice
way to wrap the long lines, so add a helper function

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7065f529 by Tim Beale at 2018-08-17T00:58:28Z
Refactor for PEP8 warning E501 line too long

Rename a parameter that an internal function takes so that the function
call doesn't overrun 80 chars.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
68f8a1c2 by Tim Beale at 2018-08-17T00:58:28Z
Fix PEP8 warning E501 line too long

Mostly involves splitting up long strings or comments so that they
span multiple lines. Some place-holder variables have been added in a
few places to avoid exceeding 80 chars.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
60b4a1be by Tim Beale at 2018-08-17T00:58:28Z
Refactor for PEP8 warning E501 line too long

Add a wrapper function to avoid long lines. This also helps
a little to manage/contain the complexity of the code.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
f9ff50ae by Tim Beale at 2018-08-17T00:58:28Z
Refactor for PEP8 warning E501 line too long

Rename a couple of really long functions.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
b0130fe4 by Andrew Bartlett at 2018-08-17T03:53:54Z
docs smb.conf: Clarify that wreplsrv:periodic_interval is in seconds

As requested by oota on samba-technical

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Aug 17 05:53:54 CEST 2018 on sn-devel-144

- - - - -
fc41281e by Volker Lendecke at 2018-08-17T05:39:16Z
torture3: Simplify the g_lock6 test

Do string_term_tdb_data just once, this is a leftover from a sweeping
change from "char *" to TDB_DATA as g_lock key.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13195
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8734970c by Volker Lendecke at 2018-08-17T05:39:17Z
torture3: Enhance g_lock6 to run without CLEAR_IF_FIRST

CLEAR_IF_FIRST doesn't really work in the cluster. This needs to be
applied to all tests, but lock6 is what I care about right now.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13195
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
682fafe2 by Volker Lendecke at 2018-08-17T08:34:53Z
torture3: Extend the g_lock6 test to also cover upgrades

The fixes for #13195 were incomplete and did not cover upgrades
properly. It's all gone in master with the new code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13195
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Aug 17 10:34:53 CEST 2018 on sn-devel-144

- - - - -
0f6b4b43 by Volker Lendecke at 2018-08-17T09:30:10Z
pygpo: Fix a talloc_tos() leak in py_gpo_get_unix_path

cache_path() implicitly puts its result on talloc_tos(). As in
py_gpo_get_unix_path the talloc_stackframe() is only created after the
cache_path() call, we leak the result of cache_path() on
talloc_tos() (which might or might not exist).

This converts the function to the pattern used elsewhere: Create the
stackframe as the very first action and remove it as the very last
action in the function.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
0a9273f8 by Volker Lendecke at 2018-08-17T09:30:10Z
lib: Pass "mem_ctx" down to xx_path

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
f986a73b by Volker Lendecke at 2018-08-17T09:30:10Z
lib: Pass mem_ctx to lock_path()

Fix a confusing API: Many places TALLOC_FREE the path where it's not
clear you have to do it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
c2ea1007 by Volker Lendecke at 2018-08-17T09:30:11Z
lib: Pass mem_ctx to state_path()

Fix a confusing API: Many places TALLOC_FREE the path where it's not
clear you have to do it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
6ca5ba52 by Volker Lendecke at 2018-08-17T12:28:51Z
lib: Pass mem_ctx to cache_path()

Fix a confusing API: Many places TALLOC_FREE the path where it's not
clear you have to do it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Aug 17 14:28:51 CEST 2018 on sn-devel-144

- - - - -
29eb2fc6 by Volker Lendecke at 2018-08-17T16:25:08Z
smbd: Enhance debugging in set_file_size

I've stumbled over a case where VFS_FTRUNCATE wasn't called due to an
unchanged size. Make that easier to detect. Also, get rid of an ancient
cast to (double).

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
04f62984 by Volker Lendecke at 2018-08-17T16:25:08Z
smbd: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a56bd0a9 by Volker Lendecke at 2018-08-17T16:25:08Z
smbd: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fea8ebce by Volker Lendecke at 2018-08-17T16:25:08Z
g_lock: Fix DEBUG messages

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0d38e9e8 by Volker Lendecke at 2018-08-17T16:25:08Z
dbwrap: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Rowland Penny <rpenny at samba.org>

- - - - -
22be863b by Volker Lendecke at 2018-08-17T16:25:08Z
smbd: Fix a few DEBUG statements

%u is not necessarily correct for uint32_t, avoid casts

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
14f9e6f8 by Volker Lendecke at 2018-08-17T19:29:15Z
dbwrap: Clarify db_open_watched API

Point out in the API that "backend" talloc_moves into the watched
database.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Aug 17 21:29:15 CEST 2018 on sn-devel-144

- - - - -
41aa55f4 by Paulo Alcantara at 2018-08-17T23:32:25Z
s3: util: Do not take over stderr when there is no log file

In case we don't have either a /var/log/samba directory, or pass a
non-existent log directory through '-l' option, all commands that are
daemonized with '-D' option hang when executed within a subshell.

An example on how to trigger that:

  # rm -r /var/log/samba
  # s=$(nmbd -D -s /etc/samba/smb.conf -l /foo123)
  (never returns)

So, when the above command is executed within a subshell the following
happens:

  (a) Parent shell creates a pipe, sets write side of it to fd 1
    (stdout), call read() on read-side fd, forks off a new child process
    and then executes nmbd in it.
  (b) nmbd sets up initial logging to go through fd 1 (stdout) by
    calling setup_logging(..., DEBUG_DEFAULT_STDOUT). 'state.fd' is now
    set to 1.
  (c) reopen_logs() is called by the first time which then calls
    reopen_logs_internal()
  (d) in reopen_logs_internal(), it attempts to create log.nmbd file in
    /foo123 directory and fails because directory doesn't exist.
  (e) Regardless whether the log file was created or not, it calls
    dup2(state.fd, 2) which dups fd 1 into fd 2.
  (f) At some point, fd 0 and 1 are closed and set to /dev/null

The problem with that is because parent shell in (a) is still blocked in
read() call and the new write side of the pipe is now fd 2 -- after
dup2() in (e) -- and remains unclosed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13578

Signed-off-by: Paulo Alcantara <palcantara at suse.de>
Reviewed-by: Jim McDonough <jmcd at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Aug 18 01:32:25 CEST 2018 on sn-devel-144

- - - - -
8262efbc by Swen Schillig at 2018-08-18T00:01:26Z
ctdb: Replace calculation of bytes to read from socket by MIN() macro

The calculation of the bytes to read from the socket can be done easier
by the usage of the common MIN() macro.

Signed-off-by: Swen Schillig <swen at vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d0ed4a53 by Swen Schillig at 2018-08-18T02:58:05Z
ctdb: calculate queue input buffer size correctly

The queue's input buffer is calculated in an iterative way.
This can result in a few back and forth jumping and
a few memory allocations and mem-free cycles.
This is very time consuming and not required, because the required
memory size can be calculated right away.

Signed-off-by: Swen Schillig <swen at vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Aug 18 04:58:05 CEST 2018 on sn-devel-144

- - - - -
9ee4d946 by Andreas Schneider at 2018-08-18T13:21:39Z
python: Fix print in dns_invalid.py

https://bugzilla.samba.org/show_bug.cgi?id=13580

Signed-off-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Sat Aug 18 15:21:39 CEST 2018 on sn-devel-144

- - - - -
51d57073 by Volker Lendecke at 2018-08-20T21:28:25Z
vfs_fruit: Fix a leak of "br_lck"

Fix a panic if fruit_access_check detects a locking conflict.

do_lock() returns a valid br_lck even in case of a locking conflict.
Not free'ing it leads to a invalid lock order panic later, because
"br_lck" corresponds to a dbwrap lock on brlock.tdb.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13584

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ec3c37ee by Volker Lendecke at 2018-08-21T00:33:04Z
torture: Demonstrate the invalid lock order panic

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13584

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Aug 21 02:33:05 CEST 2018 on sn-devel-144

- - - - -
7460d9b9 by Volker Lendecke at 2018-08-21T22:57:31Z
smbd: Remove koplocks->contend_level2 callbacks

This was only implemented by onefs in this way. If we get around to use
for example fanotify or something similar, we can either re-add them or
do it in a different way. For now, simplify the code.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Wed Aug 22 00:57:31 CEST 2018 on sn-devel-144

- - - - -
f16d9172 by Volker Lendecke at 2018-08-21T22:58:41Z
libgpo: Fix CID 1438462 Error handling issues (CHECKED_RETURN)

Yes, this creates a leak of "data", but the other error exits in this
function are the same.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
09acfb76 by Volker Lendecke at 2018-08-21T22:58:41Z
dsdb: Fix CID 1438461 Error handling issues (CHECKED_RETURN)

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
781faca0 by Volker Lendecke at 2018-08-21T22:58:41Z
dsdb: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
75ced0d1 by Volker Lendecke at 2018-08-21T22:58:41Z
libads: Fix an error path talloc leak

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fb81fb2d by Volker Lendecke at 2018-08-22T01:59:51Z
libads: Simplify parse_spn()

A few lines less and quite some bytes less .text

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Aug 22 03:59:51 CEST 2018 on sn-devel-144

- - - - -
9c71f61e by Jeremy Allison at 2018-08-22T19:50:41Z
s3: smbd: Ensure get_real_filename() copes with empty pathnames.

Needed for vfs_glusterfs, as Gluster requires "." not '\0'.

Based on a fix from Anoop C S <anoopcs at redhat.com>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13585

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ira Cooper <ira at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Aug 22 21:50:41 CEST 2018 on sn-devel-144

- - - - -
c39ec642 by Volker Lendecke at 2018-08-23T10:08:21Z
torture: Make sure that fruit_ftruncate only unlinks streams

Follow-up to

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13441

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
8c142348 by Volker Lendecke at 2018-08-23T13:28:47Z
vfs_fruit: Don't unlink the main file

The original fix for bug 13441 was missing a check that verifies that
fruit_ftruncate() is actually called on a stream.

Follow-up to

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13441

Pair-Programmed-With: Volker Lendecke <vl at samba.org>

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Thu Aug 23 15:28:48 CEST 2018 on sn-devel-144

- - - - -
831bb357 by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E271: multiple spaces after keyword

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e551f5f5 by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E713: test for membership should be 'not in'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5027405d by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E714: test for object identity should be 'is not'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d7863ffe by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E731: do not assign a lambda expression, use a def

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
3606a49c by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix W601: .has_key() is deprecated, use 'in'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1c5fa5dc by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix W602: deprecated form of raising exception

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
086daf5f by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E101: indentation contains mixed spaces and tabs

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
ada5af53 by Joe Guo at 2018-08-24T05:49:25Z
PEP8: whitespace fixes in wintest.py

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
4fc08d8f by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E111: indentation is not a multiple of four

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
eba81f7f by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E115: expected an indented block (comment)

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5037731e by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E116: unexpected indentation (comment)

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
b43408b3 by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E121: continuation line under-indented for hanging indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
297faf32 by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E122: continuation line missing indentation or outdented

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
484ce063 by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E123: closing bracket does not match indentation of opening bracket's line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
dba0c7eb by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E124: closing bracket does not match visual indentation

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
aa163f1a by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E125: continuation line with same indent as next logical line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
bbb9f576 by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E127: continuation line over-indented for visual indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5d532543 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E128: continuation line under-indented for visual indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
944d7043 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E131: continuation line unaligned for hanging indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a9551eda by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E201: whitespace after '('

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
562411bd by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E202: whitespace before ')'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
ba0827b5 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E203: whitespace before ':'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d9c282a9 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E211: whitespace before '('

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fb5ea356 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E222: multiple spaces after operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
32266d2d by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E225: missing whitespace around operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
87bbc2df by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E226: missing whitespace around arithmetic operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e1edeae8 by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E227: missing whitespace around bitwise or shift operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
35de4227 by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E228: missing whitespace around modulo operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
12d3fbe1 by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E231: missing whitespace after ','

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
40ef91fb by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E241: multiple spaces after ','

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1ccc36b4 by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E251: unexpected spaces around keyword / parameter equals

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
c809a860 by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E261: at least two spaces before inline comment

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7a07d422 by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E265: block comment should start with '# '

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
542e91ef by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E301: expected 1 blank line, found 0

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
211c9a5f by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E302: expected 2 blank lines, found 1

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fd6b2086 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E303: too many blank lines (2)

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
115f2a71 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E305: expected 2 blank lines after class or function definition, found 1

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
4d529c40 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E306: expected 1 blank line before a nested definition, found 0

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
95c36d82 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E401: multiple imports on one line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7b031b01 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E502: the backslash is redundant between brackets

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d23170c9 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E701: multiple statements on one line (colon)

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
cabb2997 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E703: statement ends with a semicolon

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e940e4cd by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E711: comparison to None should be 'if cond is not None:'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1676a4dc by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix E712: comparison to False should be 'if cond is False:' or 'if not cond:'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
9f5bbcc1 by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix E713: test for membership should be 'not in'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
aa243d1a by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix W291: trailing whitespace

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
6f9eb617 by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix W293: blank line contains whitespace

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
c9f2fdea by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix W391: blank line at end of file

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
4284f604 by Joe Guo at 2018-08-24T05:49:31Z
PEP8: add pycodestyle config in setup.cfg

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
9c750ad7 by Douglas Bagnall at 2018-08-24T05:49:31Z
PEP8: more space before equals

This dropped out during rebase

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2ccb0b03 by Douglas Bagnall at 2018-08-24T05:49:31Z
PEP8: improve spacing around colons

These dropped out of Joe's patches during rebase and review.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5d0dc38c by Douglas Bagnall at 2018-08-24T05:49:31Z
ldb tests: remove unused code from match_rules.py

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
333bc327 by Douglas Bagnall at 2018-08-24T05:49:31Z
PEP8: better formatting of (CONST1|CONST2)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
08d1573b by Douglas Bagnall at 2018-08-24T05:49:31Z
PEP8: add spaces after operators

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d246eec0 by Douglas Bagnall at 2018-08-24T05:49:32Z
dcerpc/dns_server tests: use record malformed as probably intended

Both ways the record is malformed, but it is more likely we meant
AAAAAAAAAA... 1
than
A 1A 1A 1A 1A ...

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
944a4609 by Douglas Bagnall at 2018-08-24T05:49:32Z
dcerpc py tests: improve argument formatting

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
06320851 by Douglas Bagnall at 2018-08-24T05:49:32Z
PEP8: improve formatting around bit-wise OR ("|")

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
aa5db3da by Douglas Bagnall at 2018-08-24T05:49:32Z
PEP8: improve formatting around various operators

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0f3f63f2 by Douglas Bagnall at 2018-08-24T08:58:16Z
PEP8: line up a couple of lists

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Aug 24 10:58:16 CEST 2018 on sn-devel-144

- - - - -
48335725 by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-common: Fix aliasing issue in IPv6 checksum

Since commit 9c51b278b1700cd5f3e2addc19b7c711cc2ea10b the compiler has
been able to inline the affected call to uint16_checksum().  Given
that the data (phdr) is being accessed by an incompatible
pointer (data) there is an aliasing problem when the call is inlined.
This results in incorrect behaviour with -O2/-O3 when compiling with
at least GCC 6, 7, and 8.

Fix this by making the types compatible.

Also fixes CID 1437604 (Reliance on integer endianness).  This is a
false positive because the uint16_checksum doesn't depend on the order
of the input uint16_t items.

https://bugzilla.samba.org/show_bug.cgi?id=13588

Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d4afb60a by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-doc: Make config migration script notice removed CTDB_BASE option

This should never have been a user-level option, but some people used
it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
64d4a7ae by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-doc: Handle boolean options in config migration more carefully

Values for ctdb.conf options are now returned by
get_ctdb_conf_option().  The main goal is to allow old boolean options
to be replaced by new logically negated options.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
17068e75 by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-config: Change option "no realtime" option to "realtime scheduling"

Negative options can be confusing, so switch to a positive option.

This was supposed to be done months ago but was forgotten.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
43adcd71 by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-doc: Change option "no realtime" option to "realtime scheduling"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8ddfc26d by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-doc: Add support for migrating tunables to ctdb.conf options

This will become common, so will be useful to have support for.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f42486e8 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-config: Switch tunable TDBMutexEnabled to a config option

Use the "database:tdb mutexes" option instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a9758f41 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-doc: Switch tunable TDBMutexEnabled to a config option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
21de59ab by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-common: Allow boolean configuration values to have yes/no values

This make the new configuration style more consistent with the old one.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
914e9f22 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-daemon: Pass DisableIPFailover tunable via environment variable

Preparation for obsoleting this tunable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8e160d33 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-tests: Drop DisableIPFailover simple test

This is about to become a config file option that can't be dynamically
changed at run-time, so drop this test for now.  This test will be added
once the tunable becomes a config file option.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
893dd623 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-failover: Add failover configuration options

Only a "disabled" option for now.  Not documented because it isn't
used yet.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d003a41a by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-config: Integrate failover options into conf-tool

Update and add tests accordingly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
92963412 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-config: Switch tunable DisableIPFailover to a config option

Use the "failover:disabled" option instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
78aad762 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-doc: Switch tunable DisableIPFailover to a config option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
55893bf8 by Martin Schwenke at 2018-08-24T08:59:22Z
ctdb-tests: Add an extra conf loading test case

This shows that config file loading continues in spite of unknown keys
if ignore_unknown is true.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
6fb80cbf by Martin Schwenke at 2018-08-24T12:13:12Z
ctdb-tests: Check that no IPs are assigned when failover is disabled

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Fri Aug 24 14:13:12 CEST 2018 on sn-devel-144

- - - - -
7c89edfe by Bernd Kuhls at 2018-08-24T15:22:09Z
Fix uClibc build on 64bit platforms by including stdint.h

Fixes an error detected by buildroot autobuilders:
http://autobuild.buildroot.net/results/573/573e2268e205e10d1352fa81122d8f225fdb4575/build-end.log

/home/rclinux/rc-buildroot-test/scripts/instance-1/output/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/stdint.h:122:27:
error: conflicting types for 'uintptr_t'
 typedef unsigned long int uintptr_t;
                           ^
In file included from ../lib/ldb/tests/ldb_msg.c:17:0:
../third_party/cmocka/cmocka.h:126:28: note: previous declaration of 'uintptr_t' was here
       typedef unsigned int uintptr_t;

The define __WORDSIZE is missing when cmocka.h decides how to
define uintptr_t, this patch includes stdint.h when needed.

Patch sent upstream:
https://lists.samba.org/archive/samba-technical/2018-January/125306.html

Signed-off-by: Bernd Kuhls <bernd.kuhls at t-online.de>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Aug 24 17:22:10 CEST 2018 on sn-devel-144

- - - - -
e5e66191 by Christof Schmitt at 2018-08-25T01:23:05Z
smbd: Remove unused KOPLOCK flags

This effectively reverts commit 17eba16b. It looks like these flags have
been introduced as part of the onefs support which has been removed
again. As there is no other use for the flags, remove them.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Sat Aug 25 03:23:05 CEST 2018 on sn-devel-144

- - - - -
a4f9e380 by Andrew Bartlett at 2018-08-25T07:00:16Z
selftest: Ensure winbindd is talking to the DC (itself) at startup

This might reduce issues with the first winbind-using test failing

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
9a19563e by Andrew Bartlett at 2018-08-25T07:00:16Z
selftest: Ensure all python3 tests are tagged (named) python3

This will help when splitting them into a distinct build.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
da6530d9 by Andrew Bartlett at 2018-08-25T07:00:16Z
selftest: Move samba.tests.dcerpc.rpcecho to chgdcpass:local

Before this patch, no python3-compatible tests ran against any of fl2000dc, vampire_2000_dc or chgdcpass
and so an autobuild of the samba-ad-dc-2-py3 environment would fail with no tests to run.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3d2baa9c by Andrew Bartlett at 2018-08-25T07:00:16Z
selftest: Move samba.tests.lsa_string to none environment, it does not contact a server

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
05389bcb by Andrew Bartlett at 2018-08-25T09:53:15Z
autobuild: Allow automatic handling of autobuild tasks ending in -py3

This will allow splitting up of the python2 and python3 tests without
duplication of this already complex file.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Aug 25 11:53:15 CEST 2018 on sn-devel-144

- - - - -
9a3b6470 by Andrew Bartlett at 2018-08-27T06:53:20Z
autobuild: Fix -py3 support to look in tasks (the table with the tests) not tasknames (the requested list)

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c833ec4c by Andrew Bartlett at 2018-08-27T10:00:11Z
travis-ci: Add python3-crypto to package list

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Aug 27 12:00:11 CEST 2018 on sn-devel-144

- - - - -
59f13347 by Jeremy Allison at 2018-08-27T15:28:24Z
s3: VFS: vfs_full_audit: Add $cwd arg to smb_fname_str_do_log().

Not yet used.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13565

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
4d72ebb8 by Jeremy Allison at 2018-08-27T18:23:55Z
s3: VFS: vfs_full_audit: Ensure smb_fname_str_do_log() only returns absolute pathnames.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13565

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Mon Aug 27 20:23:55 CEST 2018 on sn-devel-144

- - - - -
a5c4d91e by Volker Lendecke at 2018-08-27T21:09:15Z
libsmb: Add protocol-agnostic cli_read

So far only cli_pull could be called directly without looking at the
protocol. We did not have a simple read that did the right thing
depending on the protocol

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fd203725 by Volker Lendecke at 2018-08-27T21:09:15Z
libsmb: Rename cli_writeall_send/recv to cli_smb1_writeall_send/recv

Preparing a protocol agnostic writeall

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
082f60af by Volker Lendecke at 2018-08-27T21:09:15Z
libsmb: Expose protocol-agnostic cli_writeall_send/recv

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ffa1c040 by Volker Lendecke at 2018-08-28T00:03:07Z
examples: Add winexe re-implemented on current Samba libs

winexe from https://sourceforge.net/projects/winexe/ is a project
based on Samba libraries from 2012. According to the winexe git
repository the last Samba commit winexe was updated to is 47bbf9886f0c
from November 6, 2012. As winexe uses unpublished Samba internal
libraries, it broke over time.

This is a port of the winexe functionality to more modern Samba
versions. It still uses internal APIs, but it being part of the tree
means that it is much easier to keep up to date.

The Windows service files were taken literally from the original
winexe from the sourceforge git. Andrzej Hajda chose GPLv3 only and
not GPLv3+. As GPL evolves very slowly, this should not be a practical
problem for quite some time.

To build it under Linux, you need mingw binaries on your build
system. Under Debian stretch, the package names are gcc-mingw-w64 and
friends.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Aug 28 02:03:07 CEST 2018 on sn-devel-144

- - - - -
0da15fa7 by Andrew Bartlett at 2018-08-29T03:28:16Z
autobuild: Implement a split python2 and python2 build pattern

The default tasks will run the tests without --extra-python specified and
the new -py3 tasks will run the python3 tests only.

This will reduce the complexity of the build combinations

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
ce53dd9c by Andrew Bartlett at 2018-08-29T03:28:16Z
gitlab-ci: Run the new python3 autobuild tasks

These additional tasks should be less complex than the full build and help get us to
a pure python3 build eventually

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
38784f2b by Andrew Bartlett at 2018-08-29T03:28:16Z
autobuild: Reduce duplication of task list in autobuild

The defaulttasks or builddirs are often updated out of sync, which causes confusion until
it is resolved.

We simply choose "." as the builddir for the tasks that
are not in the default set.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
f9e494a6 by Andrew Bartlett at 2018-08-29T03:28:16Z
autobuild: Move ad_dc_no_nss into to samba-ad-dc-2 autobuild job

This tries to to split up the tasks more evenly and may help with the python3 tests
against this environment if started from a more isolated job.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
6b116436 by Andrew Bartlett at 2018-08-29T03:28:16Z
autobuild: Move backup/restore environments into to samba-ad-dc-2 autobuild job

This tries to to split up the tasks more evenly and may help with the python3
work by isolating them from the long samba job.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
acb8cf1e by Andrew Bartlett at 2018-08-29T03:28:17Z
travis-ci: Add py3 jobs to match new jobs in autobuild.py

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
47c14ef6 by Andrew Bartlett at 2018-08-29T03:28:17Z
autobuild: use close_fds=True to avoid *.stderr and *.stdout inheriting into every process

This closes fds other than 0, 1, 2.

This ensures only the correct *.stderr and *.stdout is attached, via
the stdout/stderr parameter to Popen(), but not every other FD
currently open in python at the time Popen is called.

For the tail invocation and other calls to Popen(), because fds 0, 1,
2 are still attached, these function as before.

Per https://docs.python.org/2.6/library/subprocess.html:

"If close_fds is true, all file descriptors except 0, 1 and
2 will be closed before the child process is executed. (Unix only)."

And regarding the passed in parameters:

"stdin, stdout and stderr specify the executed programs’ standard
input,
standard output and standard error file handles, respectively.  "
...

"With None (the default), no redirection will occur;
the child’s file handles will be inherited from the parent. "

(The unwanted inherited files would be on a random high FD, where the
program wouldn't know what to do with them, but counting towards the
process FD limit).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13591

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
825d67fd by Andrew Bartlett at 2018-08-29T06:20:55Z
autobuild: Avoid subshell for tail -f invocation

This should mean one less process in the process tree, and less places to hold
FDs open.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13591

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Aug 29 08:20:55 CEST 2018 on sn-devel-144

- - - - -
81f4971f by Andreas Schneider at 2018-08-29T16:33:19Z
s4:torture: Improve torture_libsmbclient_init_context()

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
30ee1d3d by Andreas Schneider at 2018-08-29T16:33:19Z
selftest: Schedule libsmbclient tests against ad_dc

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7a7d95fc by Andreas Schneider at 2018-08-29T19:28:41Z
s4:torture: Add a test for listing shares

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Aug 29 21:28:41 CEST 2018 on sn-devel-144

- - - - -
99d6237a by Alexander Bokovoy at 2018-08-30T02:48:00Z
pidl/tests: fix ndr_push_init_ctx() usage

The code in libndr got rid of the second argument quite some time ago
(2010): f9ca9e46ad24036bf00cb361a6cef4b2e7e98d7d

However, pidl tests did include a code that passed two arguments.
Recently GCC started to fail to compile such code.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Aug 30 04:48:00 CEST 2018 on sn-devel-144

- - - - -
dc6040c1 by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-common: Add support for sock daemon to notify of successful startup

The daemon writes 0 into the specified file descriptor when it is up
and listening.  This can be used to avoid loops in clients that
attempt to connect until they succeed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
11ee92d1 by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-event: Add support to eventd for the startup notification FD

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e357b62f by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-daemon: Improve error handling consistency

Other errors free argv, so do it here too.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c446ae5e by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-daemon: Open eventd pipe earlier

The pipe will soon be needed earlier, so initialise it earlier.
Ensure the file descriptors are closed on error.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
62ec1ab1 by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-daemon: Wait for eventd to be ready before connecting

The current method of retrying the connection to eventd means that
messages get logged for each failure.

Instead, pass a pipe file descriptor to eventd and wait for it to
write 0 to the pipe to indicate that it is ready to accept client
connections.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
b430a1ac by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-daemon: Do not retry connection to eventd

Confirmation is now received from eventd that it is accepting
connections, so this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
03259348 by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-common: Fix CID 1414745 - Out-of-bounds access

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
7c361f48 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Restore dropped copyright attributions

Commit fa94a49dbbec4a65c368a533a534f952a9f147a7 accidentally dropped
some copyright attributions.  The original version of system_socket.c
was based on system_linux.c but many parts have been taking from
system_freebsd.c, which had these additional copyright attributions.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0927b382 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-tests: Add basic test to sanity check types in socket marshalling

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
172b87cb by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Initialise structures when declared

Instead of using ZERO_STRUCT().

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2ebb25df by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Factor out common ARP code

Finding the interface and the MAC address are obvious.  Might as well
set up the common parts of the destination address structure.

Continue to open the socket and find the MAC address first.  This
might seem odd because marshalling and other subsequent steps may
fail.  However, in the future this code might be optimised to open a
single socket to send ARPs for a list of addresses on each interface,
so don't change the logic.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
50a6d152 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Fix error handling when sending ARPs

There are numerous places in the code where errno can be lost causing
the wrong error to be printed by a caller.  Change ctdb_sys_send_arp()
to always return a useful errno on error instead of returning -1 and
sometimes having errno set correctly.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
39cfd511 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Separate ARP and IPv6 NA marshalling code

This can be tested separately.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
87088af6 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-tests: Add tests for ARP and IPv6 NA marshalling

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e2a00fec by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Be more careful with packet sizes

Ethernet packets must be at least 64 bytes.

For ARP the packet size was limited to 64 bytes.  This is probably OK
but the code might as well be a little more general.

For IPv6 NA there was no guarantee that the packet is at least 64
bytes.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
6b1e9a43 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Use struct ether_arp to avoid manual offset calculations

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
ca0db67d by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Clarify offset and packet length calculations

Calculate each offset from the beginning of the buffer and explicitly
use the sizes of structures.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
af5a42bf by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Set version more obviously in IPv6 NA packet

Version is the top 4 bits of this field.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a6789957 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Avoid single line multi-assignment

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d7d23e78 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Factor out TCP packet marshalling code

This can be tested separately.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a02cba1c by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-tests: Add tests for TCP packet marshalling

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8fcf1af5 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Avoid magic numbers when building TCP packets

Most packet sizes and offsets are multiples of 32-bit words.  The IPv6
payload length is in octets.  The IPv6 version is the top 4 bits of
the relevant field.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0beb16f3 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Don't modify a const argument

The current code might be slightly more efficient but
intentionally (although temporarily) modifying a const argument just
seems wrong.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f3a1f1e1 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Fix a bug in non-Linux (PCAP) TCP packet capturing

Captured packets include a link-layer header, which is considered in
the Linux code but not the PCAP code.  Also, the actual captured
length is in caplen, not len.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
cb4848e3 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Fix error handling when parsing TCP packets

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
028fdc12 by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-common: Clean up types/declarations in TCP socket reading

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e2ac3686 by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-common: Factor out TCP packet parsing code

This can be tested separately.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
43a20225 by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-tests: Extend TCP packet test to also do packet extraction

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
924a655b by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-common: Improve TCP packet size and offset calculations

The IPv4 check for short packets was strange.  It appeared to ensure
that the capture included everything up to and including the window
size.  The checksum field immediately follows the window size field,
so just ensure that the packet is large enough to contain everything
up to the start of the checksum.

Add a similar check for IPv6 packets.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
53ceac96 by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-common: Check the version field in IPv6 packets

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
58b8f2a3 by Martin Schwenke at 2018-08-30T05:50:04Z
ctdb-common: Clean up comments in TCP packet parsing

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Thu Aug 30 07:50:04 CEST 2018 on sn-devel-144

- - - - -
9f60a77e by Christof Schmitt at 2018-08-30T16:07:14Z
lib: Fix lstat check in directory_create_or_exist

The lstat check in directory_create_or_exist did not verify whether an
existing object is actually a directory. Also move the check to only
apply when mkdir returns EEXIST; this fixes CID 241930 Time of check
time of use.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
83f0ac5c by Christof Schmitt at 2018-08-30T19:19:31Z
torture: Add test for directory_create_or_exist

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Thu Aug 30 21:19:31 CEST 2018 on sn-devel-144

- - - - -
b7d77ce4 by Volker Lendecke at 2018-08-31T16:42:31Z
lib: Fix the build on FreeBSD

FreeBSD needs to explicitly #include <unistd.h> for geteuid() and close()

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Aug 31 18:42:31 CEST 2018 on sn-devel-144

- - - - -
44840ba5 by Ralph Boehme at 2018-08-31T20:22:22Z
vfs_delay_inject: adding delay to VFS calls

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3255822f by Ralph Boehme at 2018-08-31T20:22:22Z
s4:selftest: reformat smb2_s3only list

No change besides reformatting the list to one entry per line.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5508024a by Ralph Boehme at 2018-08-31T20:22:23Z
selftest: add a durable handle test with delayed disconnect

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b70b8503 by Ralph Boehme at 2018-08-31T20:22:23Z
s3:smbd: reorder tcon global record deletion and closing files of a tcon

As such, this doesn't change overall behaviour, but in case we ever add
semantics acting on tcon record changes via an API like
dbwrap_watch_send(), this will make a difference as it enforces
ordering.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8f6edcc1 by Ralph Boehme at 2018-08-31T20:22:23Z
s3:smbd: let session logoff close files and tcons before deleting the session

This avoids a race in durable handle reconnects if the reconnect comes
in while the old session is still in the tear-down phase.

The new session is supposed to rendezvous with and wait for destruction
of the old session, which is internally implemented with
dbwrap_watch_send() on the old session record.

If the old session deletes the session record before calling
file_close_user() which marks all file handles as disconnected, the
durable handle reconnect in the new session will fail as the records are
not yet marked as disconnected which is a prerequisite.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5d95f79f by Ralph Boehme at 2018-08-31T23:26:35Z
s3:smbd: add a comment stating that file_close_user() is redundant for SMB2

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Sep  1 01:26:35 CEST 2018 on sn-devel-144

- - - - -
cfb19e98 by Andrew Bartlett at 2018-09-03T01:22:19Z
selftest: Set NSS_WRAPPER_HOSTS when creating the trusts

Otherwise this relies on the order that tests run to cause the environment variable
to be left behind.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
85ef0590 by Andrew Bartlett at 2018-09-03T01:22:20Z
selftest: Set RESOLV_WRAPPER_CONF/RESOLV_WRAPPER_HOSTS when running dcpromo

Otherwise this relies on the order that tests run to cause the environment variable
to be left behind.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
ac30d7b6 by Andrew Bartlett at 2018-09-03T01:22:20Z
selftest: Remove excption for NSS_WRAPPER_HOSTS and RESOLV_WRAPPER_HOSTS

These must be set correctly for each command in provision also.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
45c0ec85 by Noel Power at 2018-09-03T01:22:20Z
python/samba: Fix py2/3 relative module import issue

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9688480d by Noel Power at 2018-09-03T01:22:20Z
python/samba: port open to io.open for PY2/PY3 compat

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
596db3a5 by Noel Power at 2018-09-03T01:22:20Z
python/samba: open files in binary mode as appropriate

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
27df0e81 by Noel Power at 2018-09-03T01:22:20Z
python/samba: port changes to allow samba.tests.dsdb_lock to work with PY3/PY2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
999b6db6 by Noel Power at 2018-09-03T01:22:20Z
s4/selftest: enable samba.tests.dsdb_lock for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
48f3801c by Noel Power at 2018-09-03T01:22:20Z
python/samba/tests: port samba.tests.unix for py3

L for long literal is no longer valid in Python3, also
long keyword has been removed. int and long have been unified
in python3. A duplicated test has been removed (for long and
int) only the int test remains now.
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5efb5ad9 by Noel Power at 2018-09-03T01:22:21Z
s4/selftest: enable samba.tests.dcerpc.unix for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0e53629f by Noel Power at 2018-09-03T01:22:21Z
s4/selftest/tests: enable samba.tests.samba_tool.timecmd for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
563c10ee by Noel Power at 2018-09-03T01:22:21Z
python/samba/tests: Use io.StringIO for py3 and StringIO.StrinIO for py2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
94c23ee0 by Noel Power at 2018-09-03T01:22:21Z
python/compat: use cStringIO instead of StringIO in compat.py

Using cStringIO for py2 seems to incur alot less problems and less changes
to the py2/py3 code then using StringIO.StringIO

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c6100d77 by Noel Power at 2018-09-03T01:22:21Z
python/samba: changes to make samba.tests.samba_tool.join run under py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bff4d80a by Noel Power at 2018-09-03T01:22:21Z
s4/selftest: Enable samba.tests.samba_tool.join for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
471163e8 by Noel Power at 2018-09-03T01:22:21Z
python/samba: changes needed for samba.tests.samba_tool.visualize PY2/PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e59ed0a3 by Noel Power at 2018-09-03T01:22:21Z
enable samba_tool.visualize_drs samba_tool.visualize_drs for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a701a253 by Noel Power at 2018-09-03T01:22:22Z
python/samba/tests: Port fsmo test to python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f0f1872e by Noel Power at 2018-09-03T01:22:22Z
s3/selftest: Enable samba.tests.samba_tool.fsmo for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
783d8565 by Noel Power at 2018-09-03T01:22:22Z
s4/selftest: enable samba.tests.samba_tool.processes for PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
210b950c by Noel Power at 2018-09-03T01:22:22Z
python/samba: Add some helper functions to encode/decode strings

Sometimes in PY3 we have variables whose content can be 'bytes' or
'str' and we can't be sure which. Generally this is because the
code variable to be used can be initialised (or reassigned) a value
from different api(s) or functions depending on complex conditions or
logic. Or another common case is in PY2 the variable is 'type <str>'
and in PY3 it is 'class <str>' but the function to use e.g. b64encode
requires 'bytes' in PY3. In such cases in PY3 it would be nice to avoid
excessive testing in the client code, in PY2 we would like to ensure the
code runs unchanged. In otherwords we wish to avoid in PY2 unnecessary
decode() calls which convert otherwise 'str' types to 'unicode'.
Mixing 'str' and 'unicode' can cause strange and unexpected problems,
sometimes we can't avoid this but it we should avoid if we can.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
638476f8 by Noel Power at 2018-09-03T01:22:22Z
python/samba: PY3 port for samba.tests.samba_tool.user_wdigest test

In addition to the attributes that caused some issues specifically
with the test some other ldb.bytes objects (those seen to be
used as strings) have been adjusted (with str()) to ensure they should
work correct in PY3.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d29fdb02 by Noel Power at 2018-09-03T01:22:22Z
s4/selftest: enable samba.tests.samba_tool.user_wdigest for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
765b1762 by Noel Power at 2018-09-03T01:22:22Z
s4/selftest: enable samba.tests.samba_tool.user_check_password_script py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dd1fce7c by Noel Power at 2018-09-03T01:22:22Z
python/sambe/tests/samba_tool: PY3 port for tests.samba_tool.group

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a167c449 by Noel Power at 2018-09-03T01:22:23Z
s4/selftest: enable samba.tests.group for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4764ac05 by Noel Power at 2018-09-03T01:22:23Z
s4/selftest: enable samba.tests.samba_tool.ou for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
683d51fb by Noel Power at 2018-09-03T01:22:23Z
python/samba/netcmd: changes for samab.tests.samba_tool.computer

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
84c890b2 by Noel Power at 2018-09-03T01:22:23Z
python/samba/provision: remove use of str() func for binary data

Python 2 code works with str(policy["nTSecurityDescriptor"]) however
this cannot work with Python 3. One could argue even the str method
doesn't make sense at all (returning a string) for data.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7a2df705 by Noel Power at 2018-09-03T01:22:23Z
s4/selftest: enable samba.tests.samba_tool.ntacl for Py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
252950ac by Noel Power at 2018-09-03T01:22:23Z
python/samba/netcmd: Fix password usage for py2/py3 compatability

getpass returns str (e.g. bytes) in python2 and str (unicode) in
py3. Adapt code to so we don't do illegal things (like try and decode)
a string in python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4188a02b by Noel Power at 2018-09-03T01:22:23Z
s4/selftest: Enable samba.tests.samba_tool.provision_password_check for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f86693c8 by Noel Power at 2018-09-03T01:22:23Z
python/samba/tests: Py2/Py2 enable samba.tests.samba_tool.help to run

Ensure bytes output is converted to text type for PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9bded41d by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: enable samba.tests.samba_tool.help for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ffdbc4aa by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: Enable samba.tests.samba_tool.sites for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f304a273 by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: enable samba.tests.samba_tool.dnscmd for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
945359bc by Noel Power at 2018-09-03T01:22:24Z
selftest: Update known fail with py3 variant of samba.tests.dcerpc.dnsserver

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
179cd3db by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: enable samba.tests.dcerpc.dnsserver for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b3a5dac3 by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: enable samba.tests.dcerpc.dnsserver(ad_dc) for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c958a043 by Noel Power at 2018-09-03T01:22:24Z
python/samba/tests: ensure byte content (not strings)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a467dc2a by Noel Power at 2018-09-03T01:22:25Z
python/samba/test/dcerpc: Py3 port of samba.tests.dcerpc.raw_protocol

Port code to allow this test run with either py2 or py3
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f5726ddc by Noel Power at 2018-09-03T01:22:25Z
s4/selftest: enable samba.tests.dcerpc.raw_protocol for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9df799d1 by Noel Power at 2018-09-03T01:22:25Z
python/samba/tests: PY2/PY3 required changes for samba.tests.samba_tool.user

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c6fd15e2 by Noel Power at 2018-09-03T01:22:25Z
python/samba/tests: Adjust teardown so second run of tests succeeds

A second run of this test (e.g. with --extra-python) or even
an manual re-run and the cache is still there.
use of '--cache-ldb-initialize'  will fail and so with the test
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
17290a37 by Noel Power at 2018-09-03T01:22:25Z
s4/selftest: enable samba.tests.samba_tool.user for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
82408fdc by Noel Power at 2018-09-03T01:22:25Z
python/samba/netcmd: PY2/PY3 changes required for user_virtualCryptSHA test

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9eb26065 by Noel Power at 2018-09-03T01:22:25Z
s4/selftest: enable samba.tests.samba_tool.user_virtualCryptSHA for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a740a613 by Andrew Bartlett at 2018-09-03T01:22:25Z
selftest: Prepare to allow override of lockout duration in password_lockout tests

This will make it easier to avoid flapping tests.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
74357bf3 by Andrew Bartlett at 2018-09-03T04:14:55Z
selftest: Split up password_lockout into tests with and without a call to sleep()

This means we can have a long observation window for many of the tests and
so make them much more reliable.  Many of these cause frustrating flapping
failures in our CI systems.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Sep  3 06:14:55 CEST 2018 on sn-devel-144

- - - - -
6d3d9a85 by Martin Schwenke at 2018-09-03T08:52:10Z
ctdb-daemon: Log complete eventd startup command

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
b5453bc2 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-daemon: Drop incorrect log message

The message is incorrect because the actual failure was loading the
config file.  Instead of fixing the message, drop it because
ctdb_config_load() already logs the failure.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
421d828f by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Fix log message for conf option with unknown section

This covers both options that appear before a section and options in
unknown sections.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
ebb28c57 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Log a message for unknown conf option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a017d318 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Log a message when an invalid conf value is encountered

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f1084400 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Avoid ENOENT for unknown conf type tags

Only use ENOENT for missing configuration file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
920ed66b by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Avoid ENOENT for unknown conf options

Only use ENOENT for missing configuration file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4f1727fe by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Process the whole config file even if an error occurs

At the moment multiple errors will be encountered one at a time, on
each load or validate.  Instead, allow all configuration errors to
printed in a single pass.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
eed738a3 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-tests: If bin/ isn't in ctdb/ then look one level higher

CTDB's test suite doesn't work from a top-level compile.  The first
step to fixing this is to correctly locate the bin/ directory.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8aacde3c by Martin Schwenke at 2018-09-03T08:52:12Z
ctdb-tests: Use known install paths in local daemon tests

The in-tree local daemons tests don't work from a top-level Samba
compile.  The simple test suite was the first test suite and things
have generally worked, so it has been slow to adopt general test
infrastructure changes.

Instead of re-calculating script and helper locations, use the paths
from script_install_paths.sh.  The bin/ directory is already added to
PATH in common.sh, so don't add it here.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
30eb2881 by Martin Schwenke at 2018-09-03T12:04:00Z
ctdb-tests: Don't run valgrind or other tracing in simple_test_command()

This function is used to run a extra command to check a result.  This
command is usually a script (often a stub) or an external command, so
no need to trace it with valgrind or whatever else might be specified.
In the worst case the command being run is a shell function, which
valgrind won't be able to find.

There is little use running the event script tests under valgrind.
However, when the whole test suite is being run under valgrind then it
should work.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Sep  3 14:04:00 CEST 2018 on sn-devel-144

- - - - -
0bd109b7 by Volker Lendecke at 2018-09-03T16:44:23Z
smbd: Fix a memleak in async search ask sharemode

fetch_share_mode_unlocked_parser() takes a "struct
fetch_share_mode_unlocked_state *" as
"private_data". fetch_share_mode_send() used a talloc_zero'ed "struct
share_mode_lock". This lead to the parser putting a "struct
share_mode_lock on the NULL talloc_context where nobody really picked it
up.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13602
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
89f9c163 by Volker Lendecke at 2018-09-03T16:44:23Z
dbwrap_tool: Simplify dbwrap_tool_erase

That's what dbwrap_wipe is for :-)

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
46819e26 by Volker Lendecke at 2018-09-03T16:44:24Z
dbwrap_tool: Simplify listkey_fn

To me dbwrap_record_get_key(rec).dsize just looks a bit ugly

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
02d44842 by Volker Lendecke at 2018-09-03T16:44:24Z
dbwrap_tool: Simplify persistent/non-persistent check

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
0ce26c75 by Volker Lendecke at 2018-09-03T16:44:24Z
dbwrap_tool: Avoid an unnecessary "else"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
05e618cb by Volker Lendecke at 2018-09-03T19:38:40Z
dbwrap_tool: We don't do "listwatchers" anymore

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Mon Sep  3 21:38:40 CEST 2018 on sn-devel-144

- - - - -
dffc182c by Stefan Metzmacher at 2018-09-04T00:31:26Z
s4:torture/rpc/netlogon: assert that cli_credentials_get_{workstation,password} don't return NULL

This is better that generating a segfault while dereferencing a NULL
pointer later.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d5dd8fdc by Stefan Metzmacher at 2018-09-04T00:31:26Z
s4:torture/rpc/netlogon: verify the trusted domains output of LogonGetDomainInfo()

This makes sure we don't treat trusted domains in the same way we treat
our primary domain.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f5f96f55 by Stefan Metzmacher at 2018-09-04T00:31:26Z
dsdb/util_trusts: domain_dn is an input parameter of dsdb_trust_crossref_tdo_info()

We should not overwrite it within the function.
Currently it doesn't matter as we don't have multiple domains
within our forest, but that will change in future.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c1b0ac95 by Stefan Metzmacher at 2018-09-04T00:31:26Z
dsdb:util_trusts: add dsdb_trust_local_tdo_info() helper function

This is similar to dsdb_trust_xref_tdo_info(), but will also work
if we ever support more than one domain in our forest.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0e442e09 by Stefan Metzmacher at 2018-09-04T00:31:27Z
s4:dsdb/common: add samdb_domain_guid() helper function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
61333f77 by Stefan Metzmacher at 2018-09-04T00:31:27Z
s4:rpc_server/netlogon: use samdb_domain_guid()/dsdb_trust_local_tdo_info() to build our netr_OneDomainInfo values

The logic for constructing the values for our own primary domain differs
from the values of trusted domains. In order to make the code easier to
understand we have a new fill_our_one_domain_info() helper that
only takes care of our primary domain.

The cleanup for the trust case will follow in a separate commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ef0b489a by Stefan Metzmacher at 2018-09-04T00:31:27Z
s4:rpc_server/netlogon: make use of talloc_zero_array() for the netr_OneDomainInfo array

It's much safer than having uninitialized memory when we hit an error
case.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2099add0 by Stefan Metzmacher at 2018-09-04T00:31:27Z
s4:rpc_server/netlogon: don't treet trusted domains as primary in LogonGetDomainInfo()

We need to handle trusted domains differently than our primary
domain. The most important part is that we don't return
NETR_TRUST_FLAG_PRIMARY for them.

NETR_TRUST_FLAG_{INBOUND,OUTBOUND,IN_FOREST} are the relavant flags
for trusts.

This is an example of what Windows returns in a complex trust
environment:

     netr_LogonGetDomainInfo: struct netr_LogonGetDomainInfo
        out: struct netr_LogonGetDomainInfo
            return_authenticator     : *
                return_authenticator: struct netr_Authenticator
                    cred: struct netr_Credential
                        data                     : f48b51ff12ff8c6c
                    timestamp                : Tue Aug 28 22:59:03 2018 CEST
            info                     : *
                info                     : union netr_DomainInfo(case 1)
                domain_info              : *
                    domain_info: struct netr_DomainInformation
                        primary_domain: struct netr_OneDomainInfo
                            domainname: struct lsa_StringLarge
                                length                   : 0x0014 (20)
                                size                     : 0x0016 (22)
                                string                   : *
                                    string                   : 'W2012R2-L4'
                            dns_domainname: struct lsa_StringLarge
                                length                   : 0x0020 (32)
                                size                     : 0x0022 (34)
                                string                   : *
                                    string                   : 'w2012r2-l4.base.'
                            dns_forestname: struct lsa_StringLarge
                                length                   : 0x0020 (32)
                                size                     : 0x0022 (34)
                                string                   : *
                                    string                   : 'w2012r2-l4.base.'
                            domain_guid              : 0a133c91-8eac-4df0-96ac-ede69044a38b
                            domain_sid               : *
                                domain_sid               : S-1-5-21-2930975464-1937418634-1288008815
                            trust_extension: struct netr_trust_extension_container
                                length                   : 0x0000 (0)
                                size                     : 0x0000 (0)
                                info                     : NULL
                            dummy_string2: struct lsa_StringLarge
                                length                   : 0x0000 (0)
                                size                     : 0x0000 (0)
                                string                   : NULL
                            dummy_string3: struct lsa_StringLarge
                                length                   : 0x0000 (0)
                                size                     : 0x0000 (0)
                                string                   : NULL
                            dummy_string4: struct lsa_StringLarge
                                length                   : 0x0000 (0)
                                size                     : 0x0000 (0)
                                string                   : NULL
                            dummy_long1              : 0x00000000 (0)
                            dummy_long2              : 0x00000000 (0)
                            dummy_long3              : 0x00000000 (0)
                            dummy_long4              : 0x00000000 (0)
                        trusted_domain_count     : 0x00000006 (6)
                        trusted_domains          : *
                            trusted_domains: ARRAY(6)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x000e (14)
                                        size                     : 0x0010 (16)
                                        string                   : *
                                            string                   : 'FREEIPA'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x0018 (24)
                                        size                     : 0x001a (26)
                                        string                   : *
                                            string                   : 'freeipa.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 00000000-0000-0000-0000-000000000000
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-429948374-2562621466-335716826
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000022 (34)
                                                       0: NETR_TRUST_FLAG_IN_FOREST
                                                       1: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       1: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000000 (0)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000008 (8)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       1: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x0016 (22)
                                        size                     : 0x0018 (24)
                                        string                   : *
                                            string                   : 'S1-W2012-L4'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x0036 (54)
                                        size                     : 0x0038 (56)
                                        string                   : *
                                            string                   : 's1-w2012-l4.w2012r2-l4.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : afe7fbde-af82-46cf-88a2-2df6920fc33e
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-1368093395-3821428921-3924672915
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000023 (35)
                                                       1: NETR_TRUST_FLAG_IN_FOREST
                                                       1: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       1: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000004 (4)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000020 (32)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       0: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       1: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x0006 (6)
                                        size                     : 0x0008 (8)
                                        string                   : *
                                            string                   : 'BLA'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x0010 (16)
                                        size                     : 0x0012 (18)
                                        string                   : *
                                            string                   : 'bla.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 00000000-0000-0000-0000-000000000000
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-4053568372-2049667917-3384589010
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000022 (34)
                                                       0: NETR_TRUST_FLAG_IN_FOREST
                                                       1: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       1: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000000 (0)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000008 (8)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       1: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x000c (12)
                                        size                     : 0x000e (14)
                                        string                   : *
                                            string                   : 'S4XDOM'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x0016 (22)
                                        size                     : 0x0018 (24)
                                        string                   : *
                                            string                   : 's4xdom.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 00000000-0000-0000-0000-000000000000
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-313966788-4060240134-2249344781
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000022 (34)
                                                       0: NETR_TRUST_FLAG_IN_FOREST
                                                       1: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       1: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000000 (0)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000008 (8)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       1: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x0014 (20)
                                        size                     : 0x0016 (22)
                                        string                   : *
                                            string                   : 'W2012R2-L4'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x001e (30)
                                        size                     : 0x0020 (32)
                                        string                   : *
                                            string                   : 'w2012r2-l4.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 0a133c91-8eac-4df0-96ac-ede69044a38b
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-2930975464-1937418634-1288008815
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x0000001d (29)
                                                       1: NETR_TRUST_FLAG_IN_FOREST
                                                       0: NETR_TRUST_FLAG_OUTBOUND
                                                       1: NETR_TRUST_FLAG_TREEROOT
                                                       1: NETR_TRUST_FLAG_PRIMARY
                                                       1: NETR_TRUST_FLAG_NATIVE
                                                       0: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000000 (0)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000000 (0)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       0: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x0016 (22)
                                        size                     : 0x0018 (24)
                                        string                   : *
                                            string                   : 'S2-W2012-L4'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x004e (78)
                                        size                     : 0x0050 (80)
                                        string                   : *
                                            string                   : 's2-w2012-l4.s1-w2012-l4.w2012r2-l4.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 29daace6-cded-4ce3-a754-7482a4d9127c
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-167342819-981449877-2130266853
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000001 (1)
                                                       1: NETR_TRUST_FLAG_IN_FOREST
                                                       0: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       0: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000001 (1)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000000 (0)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       0: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                        lsa_policy: struct netr_LsaPolicyInformation
                            policy_size              : 0x00000000 (0)
                            policy                   : NULL
                        dns_hostname: struct lsa_StringLarge
                            length                   : 0x0036 (54)
                            size                     : 0x0038 (56)
                            string                   : *
                                string                   : 'torturetest.w2012r2-l4.base'
                        dummy_string2: struct lsa_StringLarge
                            length                   : 0x0000 (0)
                            size                     : 0x0000 (0)
                            string                   : NULL
                        dummy_string3: struct lsa_StringLarge
                            length                   : 0x0000 (0)
                            size                     : 0x0000 (0)
                            string                   : NULL
                        dummy_string4: struct lsa_StringLarge
                            length                   : 0x0000 (0)
                            size                     : 0x0000 (0)
                            string                   : NULL
                        workstation_flags        : 0x00000003 (3)
                               1: NETR_WS_FLAG_HANDLES_INBOUND_TRUSTS
                               1: NETR_WS_FLAG_HANDLES_SPN_UPDATE
                        supported_enc_types      : 0x0000001f (31)
                               1: KERB_ENCTYPE_DES_CBC_CRC
                               1: KERB_ENCTYPE_DES_CBC_MD5
                               1: KERB_ENCTYPE_RC4_HMAC_MD5
                               1: KERB_ENCTYPE_AES128_CTS_HMAC_SHA1_96
                               1: KERB_ENCTYPE_AES256_CTS_HMAC_SHA1_96
                               0: KERB_ENCTYPE_FAST_SUPPORTED
                               0: KERB_ENCTYPE_COMPOUND_IDENTITY_SUPPORTED
                               0: KERB_ENCTYPE_CLAIMS_SUPPORTED
                               0: KERB_ENCTYPE_RESOURCE_SID_COMPRESSION_DISABLED
                        dummy_long3              : 0x00000000 (0)
                        dummy_long4              : 0x00000000 (0)
            result                   : NT_STATUS_OK

Best viewed with: git show --histogram -w

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
38e97f8b by Andreas Schneider at 2018-09-04T00:31:27Z
waf: Check for -fstack-protect-strong support

The -fstack-protector* flags are compiler only flags, don't pass them to
the linker.

https://developers.redhat.com/blog/2018/03/21/compiler-and-linker-flags-gcc/

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13601

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fc4df251 by Andreas Schneider at 2018-09-04T00:31:27Z
waf: Add -fstack-clash-protection

https://developers.redhat.com/blog/2018/03/21/compiler-and-linker-flags-gcc/

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13601

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fb047b3d by Robert Scheck at 2018-09-04T00:31:27Z
Update Samba URLs from http:// to https://

Reviewed-by: Volker Lendecke <Volker.Lendecke at SerNet.DE>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f98f8e3e by Jeremy Allison at 2018-09-04T00:31:27Z
s4: torture: Ensure all notify tests use separate directories.

Makes it much easier to find tests that error out whilst
leaving directories behind.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1799633c by Jeremy Allison at 2018-09-04T00:31:27Z
s4: torture: Ensure we can't exit without deleting our directory.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c86f6c29 by Jeremy Allison at 2018-09-04T03:23:57Z
s4: torture: Ensure we close the handle on the correct tree-id.

Otherwise we leave a directory behind on exit.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Sep  4 05:23:57 CEST 2018 on sn-devel-144

- - - - -
7356e814 by Stefan Metzmacher at 2018-09-04T08:45:10Z
s3:vfs: fix valgrind warning in SMB_VFS_{PREAD,PWRITE,FSYNC}_RECV()

tevent_req_received() destroys 'state', so we need helper variables
to hold the return value.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Tue Sep  4 10:45:10 CEST 2018 on sn-devel-144

- - - - -
80324ff6 by Volker Lendecke at 2018-09-04T22:48:17Z
examples: Fix mingw64 detection

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7df50529 by Alexander Bokovoy at 2018-09-04T22:48:17Z
s4:selftest: test kinit with the interdomain trust user account

To test it, add a blackbox test that ensures we pass a keytab-based
authentication with the trust user account for a trusted domain.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
39c281a2 by Stefan Metzmacher at 2018-09-04T22:48:17Z
samba-tool: add virtualKerberosSalt attribute to 'user getpassword/syncpasswords'

This might be useful for someone, but at least it's very useful for
tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8526feb1 by Stefan Metzmacher at 2018-09-04T22:48:17Z
testprogs/blackbox: add testit[_expect_failure]_grep() to subunit.sh

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1b31fa62 by Stefan Metzmacher at 2018-09-04T22:48:17Z
testprogs/blackbox: let test_trust_user_account.sh check the correct kerberos salt

This demonstrates the bug we currently have.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f3e349be by Alexander Bokovoy at 2018-09-05T01:57:22Z
krb5-samba: interdomain trust uses different salt principal

Salt principal for the interdomain trust is krbtgt/DOMAIN at REALM where
DOMAIN is the sAMAccountName without the dollar sign ($)

The salt principal for the BLA$ user object was generated wrong.

dn: CN=bla.base,CN=System,DC=w4edom-l4,DC=base
securityIdentifier: S-1-5-21-4053568372-2049667917-3384589010
trustDirection: 3
trustPartner: bla.base
trustPosixOffset: -2147483648
trustType: 2
trustAttributes: 8
flatName: BLA

dn: CN=BLA$,CN=Users,DC=w4edom-l4,DC=base
userAccountControl: 2080
primaryGroupID: 513
objectSid: S-1-5-21-278041429-3399921908-1452754838-1597
accountExpires: 9223372036854775807
sAMAccountName: BLA$
sAMAccountType: 805306370
pwdLastSet: 131485652467995000

The salt stored by Windows in the package_PrimaryKerberosBlob
(within supplementalCredentials) seems to be
'W4EDOM-L4.BASEkrbtgtBLA' for the above trust
and Samba stores 'W4EDOM-L4.BASEBLA$'.

While the salt used when building the keys from
trustAuthOutgoing/trustAuthIncoming is
'W4EDOM-L4.BASEkrbtgtBLA.BASE', which we handle correct.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep  5 03:57:22 CEST 2018 on sn-devel-144

- - - - -
8077f462 by Thomas Nagy at 2018-09-05T04:37:21Z
build:wafsamba: Build on waf 1.9

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5d997861 by Thomas Nagy at 2018-09-05T04:37:21Z
build:wafsamba: Update dist/distcheck commands

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9aa8f2ba by Thomas Nagy at 2018-09-05T04:37:21Z
build:wafsamba: detail where we are processing the autobuild

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
59673353 by Thomas Nagy at 2018-09-05T04:37:21Z
script/autobuild: use --out instead of -b when calling configure

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
48cf9ccd by Thomas Nagy at 2018-09-05T04:37:21Z
selftest/tests.py: Update path to waflib

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cbc65346 by Thomas Nagy at 2018-09-05T04:37:22Z
thirdparty:waf: New files for waf 1.9.10

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
247d12c4 by Thomas Nagy at 2018-09-05T04:37:22Z
build:wafsamba: Ignore cfg_file absolute paths differences

Due to build variants, cfg_file paths are written as absolute paths.

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
faef2750 by Thomas Nagy at 2018-09-05T04:37:22Z
build:wafsamba: Remove unnecessary parameters to cmd_and_log

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4e65b33c by Alexander Bokovoy at 2018-09-05T04:37:22Z
third_party:waf: update to upstream 2.0.4 release

Update third_party/waf/ to 2.0.4 to bring us closer to Python 3

This change requires a number of changes in buildtools/ too.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
65074d89 by Alexander Bokovoy at 2018-09-05T04:37:22Z
ctdb/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
efa94eb3 by Alexander Bokovoy at 2018-09-05T04:37:22Z
docs-xml/wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
18c3598b by Alexander Bokovoy at 2018-09-05T04:37:22Z
dynconfig/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b999fd3c by Alexander Bokovoy at 2018-09-05T04:37:22Z
lib/crypto/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b0a12c81 by Alexander Bokovoy at 2018-09-05T04:37:22Z
lib/crypto/wscript_configure: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0c703f1d by Alexander Bokovoy at 2018-09-05T04:37:22Z
lib/ldb/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ea578830 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/replace/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
62bb5734 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/talloc/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
15288c38 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/tdb/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d4cef3b1 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/tevent/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5f577140 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/util/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e6761832 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/util/wscript_configure: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8a741163 by Alexander Bokovoy at 2018-09-05T04:37:23Z
nsswitch/wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e1e5b57f by Alexander Bokovoy at 2018-09-05T04:37:23Z
packaging/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7f6ce8f8 by Alexander Bokovoy at 2018-09-05T04:37:23Z
pidl/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fa1ca713 by Alexander Bokovoy at 2018-09-05T04:37:23Z
python/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0de67cf1 by Alexander Bokovoy at 2018-09-05T04:37:23Z
selftest/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c898f18f by Alexander Bokovoy at 2018-09-05T04:37:23Z
source3/build/charset.py: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
91e099d6 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source3/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8b5ad021 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source3/wscript_configure_system_ncurses: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
029ac7d6 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source4/dsdb/samdb/ldb_modules/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
055aae90 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source4/heimdal_build/wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c797e921 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source4/heimdal_build/wscript_configure: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0c423a3a by Alexander Bokovoy at 2018-09-05T04:37:24Z
source4/lib/tls/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ca5008a4 by Alexander Bokovoy at 2018-09-05T04:37:24Z
testsuite/headers/wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ba03d120 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/aesni-intel/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3e71f06a by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/cmocka/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2b21e67e by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/nss_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2847a382 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/pam_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7e2deed7 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/popt/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4c441530 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/resolv_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
051e7e74 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/socket_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f14b8cb0 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/uid_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0119a879 by Alexander Bokovoy at 2018-09-05T04:37:25Z
third_party/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c73b7795 by Alexander Bokovoy at 2018-09-05T04:37:25Z
wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1dc80eb1 by Alexander Bokovoy at 2018-09-05T04:37:25Z
wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e58ca30b by Alexander Bokovoy at 2018-09-05T04:37:25Z
wscript_build_embedded_heimdal: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ef4b2d45 by Alexander Bokovoy at 2018-09-05T04:37:25Z
wscript_configure_system_mitkrb5: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c5ab9ead by Alexander Bokovoy at 2018-09-05T04:37:25Z
heimdal wscript changes

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
79c4ba26 by Alexander Bokovoy at 2018-09-05T04:37:25Z
auth/wscript: fix options use

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3b7dfc51 by Alexander Bokovoy at 2018-09-05T04:37:25Z
third_party/waf: upgrade to waf 2.0.8

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7eab91a0 by Alexander Bokovoy at 2018-09-05T04:37:25Z
buildtools/wafsamba: adopt to waf 2.0.8

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
69c655fe by Alexander Bokovoy at 2018-09-05T04:37:25Z
buildtools/wafsamba: reduce imports

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3fc47866 by Alexander Bokovoy at 2018-09-05T04:37:25Z
buildtools/wafsamba: add install_dir to build context

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
944fce25 by Alexander Bokovoy at 2018-09-05T04:37:25Z
auth/wscript: import from waflib

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d00ba400 by Alexander Bokovoy at 2018-09-05T04:37:25Z
nsswitch/libwbclient/wscript: import from waflib

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9f2f5b4d by Alexander Bokovoy at 2018-09-05T04:37:26Z
source3/libsmb/wscript: remove unneeded import

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7aaa1e68 by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba/samba_abi: always_run helper was deprecated in waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
175be937 by Alexander Bokovoy at 2018-09-05T04:37:26Z
ctdb/wscript: adopt to waf-2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2e401d24 by Alexander Bokovoy at 2018-09-05T04:37:26Z
wscript: adopt to waf-2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
850ceec8 by Thomas Nagy at 2018-09-05T04:37:26Z
buildtools/wafsamba: compile asn1 files by adding missing code from compat15

Signed-off-by:    Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1fdcbd06 by Alexander Bokovoy at 2018-09-05T04:37:26Z
wafsamba: use correct context for APPNAME

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2a63619c by Alexander Bokovoy at 2018-09-05T04:37:26Z
waf heimdal: use absolute path to compile_et

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
35ebfd3d by Alexander Bokovoy at 2018-09-05T04:37:26Z
selftest/wscript: handle lists in environmental variables in waf

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cdda0d90 by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba: change SAMBA_BUILD_ENV to use bldnode.abspath()

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0fdba4be by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba: use top for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
699977a2 by Alexander Bokovoy at 2018-09-05T04:37:26Z
wscript: adopt to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5c3d31eb by Alexander Bokovoy at 2018-09-05T04:37:26Z
cdtb/wscript: use top and out for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fb43723b by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba: use top and out for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
aebcd698 by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba: crosscompile should use Utils.subprocess in waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1d25ae58 by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: use context instead of options for cross-compile checks for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
77f3a138 by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: use cflags instead of ccflags for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
31f8945a by Alexander Bokovoy at 2018-09-05T04:37:27Z
wafsamba: install Python modules back to bin/python, not bin/python_modules

Partially revert 80fce353e740c793619005ac102ab07fb5e7d280 which started
installing generated Python modules into python_modules directory back
in 2013. This, unfortunately, does not work anymore as Python gets quite
confused with our setup even when both bin/python and bin/python_modules
directories are part of sys.path.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
75d5bcb4 by Alexander Bokovoy at 2018-09-05T04:37:27Z
selftest/wscript: properly handle env.cwd which is a list, not a string

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2a3fcdf1 by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: generate build options output with waf 2.0

With WAF 2.0 we get all defines in environment at the same level.
Fix build options source code generator to handle this.

I felt uneasy at filtering out some defines so instead the code
is mangling generic defines to be correct for C compiler by
replacing '-', '.', and '()' with an underscore ('_').

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7944ed6e by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: port stale_files to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1efe6890 by Alexander Bokovoy at 2018-09-05T04:37:27Z
wscript: port build_system_mitkrb5 to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
72a9e6d1 by Alexander Bokovoy at 2018-09-05T04:37:27Z
wscript: port build_system_heimdal to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fdd89fef by Alexander Bokovoy at 2018-09-05T04:37:27Z
selftest/tests.py: update to support waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0a9d98ba by Alexander Bokovoy at 2018-09-05T04:37:27Z
ctdb/wscript: rework how version number is retrieved

Using default context functions before waf initialization occured
is prone to error. Postpone calling samba_version.* code until we
got default context initialized.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f495f64e by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: remove ENFORCE_GROUP_ORDERING

ENFORCE_GROUP_ORDERING is not needed with waf 2.0 anymore

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4c7c10b2 by Alexander Bokovoy at 2018-09-05T04:37:27Z
lib/audit_logging: update to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6fc9f1a8 by Alexander Bokovoy at 2018-09-05T04:37:27Z
lib/mscat: fix logging in wscript

- - - - -
72c3ff9a by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: use CACHE_SUFFIX instead of a hard-coded name

waflib.Build provides CACHE_SUFFIX constant to append to the target
name. We have a reference to samba-specific cache suffix (.cache.py)
while original WAF uses _cache.py as a cache suffix since 2011 (see
commit 44a967e326cc2e670a31b3712e4763b72d65e81b in WAF project code).

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
67ed1ea6 by Alexander Bokovoy at 2018-09-05T04:37:28Z
script/autobuild: re-use CACHE_SUFFIX from waflib

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8f022a0b by Alexander Bokovoy at 2018-09-05T04:37:28Z
script/autobuild: Fix formatting in send_email

Commit cb40e2bbc8a34a1ec3584ab585c5bf44c037ef0e introduced a print
statement with a broken formatting. Reported by pylint.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8de348e9 by Andrew Bartlett at 2018-09-05T07:29:39Z
third_party: Import exact files from waf-2.0.8/waflib

wget https://waf.io/waf-2.0.8.tar.bz2
tar -xf waf-2.0.8.tar.bz2
rsync -a waf-2.0.8/waflib/ third_party/waf/waflib/

The previous import was damaged by auto-strip/correct of whitespace
and had other small corrections.

Check with git show -w.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep  5 09:29:39 CEST 2018 on sn-devel-144

- - - - -
630cc6e6 by Andrew Bartlett at 2018-09-05T09:42:23Z
torture: Add tests to prove that kinit to an SPN is not allowed (unless it is also a UPN)

The krb5.kdc.canon testsuite has been updated to pass against Windows
Server 1709 in four modes:

* A normal user
* A user with a UPN
* A user with an SPN (machine account)
* A user with an SPN as the UPN

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
364c13ac by Andrew Bartlett at 2018-09-05T09:42:24Z
selftest/samba4.blackbox.export.keytab: Remove stray exit 0 and so run cleanup

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
a6182bd9 by Andrew Bartlett at 2018-09-05T09:42:24Z
Revert "s4/heimdal: allow SPNs in AS-REQ"

This reverts commit 20dc68050df7b1b0c9d06f8251183a0a6283fcaf.

Tests (the krb5.kdc testsuite) show this behaviour is incorrect.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
71ba7cb9 by Andrew Bartlett at 2018-09-05T09:42:24Z
selftest: Add new test to run krb5.kdc.canon against a user with an SPN for a UPN

The failures in this test compared with Windows Server 1709 are added to
knownfail.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3e5ad202 by Andrew Bartlett at 2018-09-05T09:42:25Z
selftest/samba4.blackbox.export.keytab: Update to use a principal with SPN as UPN

The ability the kinit with an SPN (not also being a UPN) has gone away as
windows doesn't offer this functionality.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
9e1ba904 by Andrew Bartlett at 2018-09-05T09:42:26Z
torture: Confirm that this element of the krb5.kdc test does not pass against Windows

This should be fixed, but in the meantime add clue to avoid regressions on
bug 11539.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
f7c409c4 by Andrew Bartlett at 2018-09-05T09:42:26Z
torture krb5.kdc.canon: Correct principal being checked in TEST_AS_REQ_SELF stage

We have already checked the client principal.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
41473daf by Andrew Bartlett at 2018-09-05T09:42:26Z
heimdal: Change KDC to respect HDB server name type if f.canonicalize is set

This changes behaviour flagged as being for Java 1.6.  My hope is that this does not
set f.canonicalize

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
aa07400f by Andrew Bartlett at 2018-09-05T14:17:59Z
kdc: Improve code clarity with extra brackets

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep  5 16:17:59 CEST 2018 on sn-devel-144

- - - - -
e0e86e8a by Andreas Schneider at 2018-09-05T16:22:24Z
smbclient: Fix errors printed if we can connect with SMB1

smbclient -L //server

before:

Reconnecting with SMB1 for workgroup listing.
Connection to earth.milkyway.site failed (Error NT_STATUS_RESOURCE_NAME_NOT_FOUND)
Failed to connect with SMB1 -- no workgroup available

after:

Reconnecting with SMB1 for workgroup listing.
Unable to connect with SMB1 -- no workgroup available

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
eddcbfea by Andreas Schneider at 2018-09-05T16:22:24Z
s3:smbclient: Do not call cli_RNetShareEnum if SMB1 is disabled

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
50f89bcd by Andreas Schneider at 2018-09-05T16:22:24Z
s3:libsmbclient: Do not call cli_RNetShareEnum if SMB1 is disabled

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0dae4e2f by Andreas Schneider at 2018-09-05T16:22:24Z
s3:libsmbclient: Add function to set protocol levels

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
931e64d2 by Andreas Schneider at 2018-09-05T16:22:24Z
s4:torture: Set credentials for libsmbclient correctly

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
778878a3 by Andreas Schneider at 2018-09-05T19:56:11Z
selftest: Run libsmbclient tests with NT1 and SMB3

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Sep  5 21:56:11 CEST 2018 on sn-devel-144

- - - - -
36c3c7d1 by Noel Power at 2018-09-05T21:27:11Z
s4/selftest: enable tdb.python for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ccbecac3 by Noel Power at 2018-09-05T21:27:11Z
python/samba/tests: make password_hash.py py2/py3 compatible

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5bb286ab by Noel Power at 2018-09-05T21:27:11Z
s4/selftest: enable samba.tests.password_hash_gpgme for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1d18ef26 by Noel Power at 2018-09-05T21:27:11Z
python/samba/tests: py3 port for samba.test.password_hash_ldap

user samba.compat.text_type instead of unicode

in py2 text_type is an alias for unicode
in py3 text_type is an alias for str (which does nothing if
the input is already str)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0854cca4 by Noel Power at 2018-09-05T21:27:12Z
s4/selftest: enable samba.tests.password_hash_fl2008 for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b7909eb2 by Noel Power at 2018-09-05T21:27:12Z
s4/selftest: enable samba.tests.password_hash_ldap for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c719b036 by Noel Power at 2018-09-05T21:27:12Z
selftest/knownfail.d: Add PY3 rule for samba.tests.encrypted_secrets

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e96fcb88 by Noel Power at 2018-09-05T21:27:12Z
python/samba/tests: PY3 port of samba.tests.encrypted_secrets

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5ffcc323 by Noel Power at 2018-09-05T21:27:12Z
s4/selftest: enable samba.tests.encrypted_secrets for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8970ec19 by Noel Power at 2018-09-05T21:27:12Z
selftest/knownfail.d: Add PY3 rule for samba.tests.py_credentials

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3fd9b77a by Noel Power at 2018-09-05T21:27:12Z
auth/credentials: py2/py3 credential key needs to return bytes

new_client_authenticator returns a dictionary. The key
'credential' needs to return bytes in Python3, without this
change the the code will attempt to convert the binary data to
a string (resulting sometimes in decode errors).

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c73c9be1 by Noel Power at 2018-09-05T21:27:12Z
python/samba/tests: port samba.tests.py_credentials for py2/py3 compat

+ ord takes 'str' param
+ unicode doesn't exist in py3
+ py3 bytes class doesn't have encode method, try to ensure py2
  code runs unchanged while adapting code (by using get_string)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9dd1271a by Noel Power at 2018-09-05T21:27:12Z
s4/selftest: enable samba.tests.py_credentials for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
94c98200 by Noel Power at 2018-09-05T21:27:13Z
python/samba/tests: fix traffic for py2/py3 compatability

load/save operations return randomly sorted dictionaries in python3
these changes make sure the already sorted expected results are
compared with sorted actual values.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2a97996c by Noel Power at 2018-09-05T21:27:13Z
python/samba/emulate: PY3 port of samba.tests.emulate.traffic_packet

Fixes
+ None cannot be used with '<' or '>' operators
+ ord expects 'str'
+ unicode doesn't exist in py3
+ bytes class does not have encode method

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5c2e06f7 by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: enable samba.tests.emulate.traffic for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6401eba1 by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: samba.tests.emulate.traffic_packet enable for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4ca09e43 by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: samba.tests.blackbox.traffic_replay enabled for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ad172841 by Noel Power at 2018-09-05T21:27:13Z
python/samba/tests: Change native string to binary for py2/py3 compat

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d0553847 by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: Enable samba.tests.blackbox.traffic_learner for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
06f43e9b by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: enable samba.tests.blackbox.traffic_summary for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
42dfc985 by Noel Power at 2018-09-05T21:27:14Z
s4/selftest: Enable samba.tests.blackbox.smbcontrol for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
05862b7b by Noel Power at 2018-09-06T00:27:51Z
s4/selftest: enable samba4.schemaInfo.python for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Sep  6 02:27:51 CEST 2018 on sn-devel-144

- - - - -
bc62182f by Martin Schwenke at 2018-09-06T06:33:59Z
ctdb-tests: Check result of write() in ARP and TCP tests

CTDB -O3 --picky-developer build is failing.  Not sure how this
slipped through.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Thu Sep  6 08:33:59 CEST 2018 on sn-devel-144

- - - - -
c4c6ca5e by Andreas Schneider at 2018-09-06T10:06:05Z
wafsamba: Support clang as C compiler

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Thu Sep  6 12:06:05 CEST 2018 on sn-devel-144

- - - - -
bc188dd7 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: ignore rules and exclude files

Will fix other ones and come back to these later.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
23b3c5b6 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: fix comment indent

The comments in this file will make different PEP8 rules fight.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d02ec9f2 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: fix E101: indentation contains mixed spaces and tabs

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0f67b6e4 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: fix E111: indentation is not a multiple of four

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
24fe8504 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: fix E122: continuation line missing indentation or outdented

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8678fde5 by Joe Guo at 2018-09-06T10:10:12Z
PEP8: fix E123: closing bracket does not match indentation of opening bracket's line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7ff6c198 by Joe Guo at 2018-09-06T10:10:12Z
PEP8: fix E124: closing bracket does not match visual indentation

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dc02a4c3 by Joe Guo at 2018-09-06T10:10:12Z
PEP8: fix E125: continuation line with same indent as next logical line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
deb81991 by Joe Guo at 2018-09-06T10:10:12Z
PEP8: fix E127: continuation line over-indented for visual indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8c2c9794 by Joe Guo at 2018-09-06T13:50:17Z
PEP8: fix E128: continuation line under-indented for visual indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Sep  6 15:50:17 CEST 2018 on sn-devel-144

- - - - -
e5afa011 by Noel Power at 2018-09-06T13:51:35Z
python/samba/tests: port samba.tests.dsdb_schema_attributes to PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
7b4db746 by Noel Power at 2018-09-06T13:51:35Z
s4/selftest: enable samba.tests.dsdb_schema_attributes for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
2b4d2810 by Noel Power at 2018-09-06T13:51:35Z
s4/torture/drs: PY3 port for samba4.drs.replica_sync_rodc

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
d18c9dc6 by Noel Power at 2018-09-06T13:51:35Z
s4/selftest: enable samba4.drs.replica_sync_rodc.python for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
59ff17cc by Noel Power at 2018-09-06T13:51:35Z
s3/selftst: enable samba4.tombstone_reanimation for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
2200b22b by Noel Power at 2018-09-06T13:51:36Z
s4/dsdb/tests: port samba4.tombstone_reanimation for PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
4a63ab95 by Noel Power at 2018-09-06T17:24:54Z
s4/selftest/tests: Enabled samba.tests.samba_tool.computer

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Thu Sep  6 19:24:54 CEST 2018 on sn-devel-144

- - - - -
e7612710 by Ralph Boehme at 2018-09-07T09:37:23Z
s3/wscript: fix bison detection

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
193fdbf1 by Ralph Boehme at 2018-09-07T09:37:23Z
s3/wscript: fix flex detection

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
311e1eb6 by Alexander Bokovoy at 2018-09-07T09:37:23Z
wafsamba/samba_autoconf: when setting undefined result, use empty tuple

A difference between waf 1.x and 2.x is that we gained 0 as an undefined
variable in the cache file. This does not allow to differentiate unset
and set to 0 defines.

Force to use empty tuple () to signify unset defines.

Also, fix handling of extra cflags in case of 'strict=True': if
extra_cflags were not defined, we'd append None to the cflags list and
it confuses conf.check() later. 'None' is added to the command line of a
tool executed by the conf.check() which, depending on a tool, may be
treated as an error and cause wrong test result.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
364077d2 by Alexander Bokovoy at 2018-09-07T09:37:23Z
lib/replace/wscript: fix detection of a fallthrough attribute for clang

clang issues a warning but otherwise allows our test to be compiled and
linked. We consider this case a successful pass for a fallthrough
attribute detection while it is an error.

Turn missing declaration warning into an error so that we actually do
not define fallthrough attribute support in case it doesn't really work.

Fixes FreeBSD 11.2 with clang.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4847daf0 by Alexander Bokovoy at 2018-09-07T13:45:36Z
wafsamba/samba_waf18: redefine flex function

There is a bug in waf: flex routine adjusts its inputs against
the task's current working directory but assumes it is being called from
within the build variant directory.

For Samba this means we adjust one level up than the actual work
directory we use to run (bin/ vs bin/default) and flex doesn't find the
source files.

Fix the issue by creating a local override of flex definition that
utilizes the same workd directory for both path adjustment and running
the flex itself.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Sep  7 15:45:36 CEST 2018 on sn-devel-144

- - - - -
d920a725 by Ralph Boehme at 2018-09-07T15:26:14Z
s3:lib/server_contexts: make server_event_ctx and server_msg_ctx static

server_event_ctx and server_msg_ctx static shouldn't be accessible from
outside this compilation unit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
dff1028e by Christof Schmitt at 2018-09-07T15:26:14Z
s3/lib:popt_common: Move setup_logging to common callback

The flag is set in the common callback, so be consistent

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
b7464fd8 by Christof Schmitt at 2018-09-07T15:26:14Z
s3:lib: Move popt_common_credentials to separate file

This is only used by command line utilities and has additional
dependencies. Move to a separate file to contain the dependencies to the
command line tools.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
8c3b62e6 by Christof Schmitt at 2018-09-07T15:26:14Z
s3:lib: Introduce cmdline context wrapper

Command line tools need acccess to the same messaging context provided
by server_messaging_context, as common code for db_open uses that
context. We want to have additional checking for command line tools
without having that code part of the servers. Introduce a wrapper
library to use for command line tools with the additional checks, that
then acquires the server_messaging_context.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
3aca3f24 by Ralph Boehme at 2018-09-07T15:26:14Z
s3:loadparm: reinit_globals in lp_load_with_registry_shares()

This was set to false in 0e0d77519c27038b30fec92d542198e97be767d9 based
on the assumption that callers would have no need to call
lp_load_initial_only() with a later call to lp_load_something().

This is not quite correct, since for accessing registry config on a
cluster with include=registry, we need messaging up and running which
*itself* requires loadparm to be initialized to get the statedir,
lockdir asf. directories.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
10e1a6eb by Ralph Boehme at 2018-09-07T15:26:15Z
selftest: pass configfile to pdbedit

This is needed otherwise pdbedit fails to initialize messaging in
autobuild.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
2c63ce94 by Ralph Boehme at 2018-09-07T15:26:15Z
s3:popt_common: use cmdline_messaging_context() in popt_common_credentials_callback()

This adds a call to cmdline_messaging_context() to the popt
popt_common_credentials_callback() hook and ensures that any client tool
that uses POPT_COMMON_CREDENTIALS gets an implicit messaging context,
ensuring it doesn't crash in the subsequent lp_load_client() with
include=registry in a cluster.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
3ecb9ed7 by Christof Schmitt at 2018-09-07T15:26:15Z
test:doc: Skip 'clustering=yes'

As testparm will error out when running clustering=yes as non-root, skip
this step to avoid a test failure.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
29fd2c2e by Christof Schmitt at 2018-09-07T15:26:15Z
s3:smbpasswd: Use cmdline_messaging_context

smbpasswd does not use POPT_CREDENTIALS. Call cmdline_messaging_context
to initialize a messaging_context with proper error checking before
calling lp_load_global.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
d7fa3815 by Christof Schmitt at 2018-09-07T15:26:15Z
s3:smbstatus: Use cmdline_messaging_context

Use cmdline_messaging_context to initialize a messaging context instead
of open coding the same steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
dd3ae2ff by Christof Schmitt at 2018-09-07T15:26:15Z
rpcclient: Use cmdline_messaging_context

Use cmdline_messaging_context with its error checking instead of open
coding the same steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f2b659e4 by Christof Schmitt at 2018-09-07T15:26:15Z
s3:net: Use cmdline_messaging_context

Use cmdline_messaging_context with its error checking instead of open
coding the same steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f56496b1 by Ralph Boehme at 2018-09-07T15:26:15Z
s3:messaging: remove unused messaging_init_client()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
4661537c by Christof Schmitt at 2018-09-07T15:26:16Z
s3:pdbedit: Use cmdline_messaging_context

Initialize the messaging context through cmdline_messaging_context to
allow access to config in clustered Samba.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
ea7a4ff7 by Christof Schmitt at 2018-09-07T15:26:16Z
s3:testparm: Use cmdline_messaging_context

Call cmdline_messaging_context to initialize a messaging config before
accessing clustered Samba config.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
cab8f27b by Christof Schmitt at 2018-09-07T15:26:16Z
s3:sharesec: Use cmdline_messaging_context

Call cmdline_messasging_context to initialize messaging context before
accessing clustered Samba config.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9ed61747 by Christof Schmitt at 2018-09-07T15:26:16Z
s3: ntlm_auth: Use cmdline_messaging_context

Call cmdline_messaging_context to initialize the messaging context
before accessing clustered Samba config.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
095123df by Christof Schmitt at 2018-09-07T15:26:16Z
s3:eventlogadm: Use cmdline_messaging_context

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
7eeff96b by Christof Schmitt at 2018-09-07T15:26:16Z
s3:dbwrap_tool: Use cmdline_messaging_context

Initialize the messaging context through cmdline_messaging_context to
allow access to config in clustered Samba.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
de040eaf by Christof Schmitt at 2018-09-07T15:26:16Z
s3:smbcontrol: Use cmdline_messaging_context

Initialize the messaging context through cmdline_messaging_context to
allow access to config in clustered Samba.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
20ed1392 by Christof Schmitt at 2018-09-07T15:26:17Z
s3:smbget: Use cmdline_messaging_context

Initialize the messaging context through cmdline_messaging_context to
allow access to config in clustered Samba.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
94852e35 by Volker Lendecke at 2018-09-07T15:26:17Z
examples: Fix the smb2mount build

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
ae7db3e5 by Christof Schmitt at 2018-09-07T15:26:17Z
s3: Rename server_event_context() to global_event_context()

This reflects that the event context is also used outside of the server
processes.

The command used for the rename:
find . -name '*.[hc]' -print0 | xargs -0 sed -i 's/server_event_context/global_event_context/'

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
cc76aaeb by Christof Schmitt at 2018-09-07T15:26:17Z
s3: Rename server_messaging_context() to global_messaging_context()

This reflects that the messaging context is also used outside of the
server processes.

The command used for the rename:
find . -name '*.[hc]' -print0 | xargs -0 sed -i 's/server_messaging_context/global_messaging_context/'

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
a7c19bca by Christof Schmitt at 2018-09-07T15:26:17Z
s3:lib/server_contexts: Rename variables

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
b88290cd by Christof Schmitt at 2018-09-07T15:26:17Z
s3:lib: Rename server_contexts to global_contexts

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
c2281341 by Volker Lendecke at 2018-09-07T15:26:17Z
smbd: Factor out file_has_read_oplocks()

Why? This makes it clearer to me that we're not interested in the actual
number of read oplocks. We only want to know if there are any read
oplocks at all.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
6a782014 by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify logic in smb2_lease_break_send

If/else if chains are hard to follow to me. Simplify the code by using
early returns.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
91085720 by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify logic in fsp_lease_update

One if-condition is better than two

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
45a2e3eb by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify lease_match() a bit

This has been implicitly initialized to 0 with the explicit struct
initializer above.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
6a76a317 by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify share_mode_stale_pid

This loop does not need to count valid share modes. A single valid one
is sufficient for keeping the delete token around

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
6be25d56 by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify logic in remove_stale_share_mode_entries

To me, an early "continue" is easier to follow than a "else".

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
a5f9b33c by Volker Lendecke at 2018-09-07T15:26:18Z
vfs_fruit: fix an uninitialized variable error

clang does not recognize "smb_panic" as an "exit()" equivalent

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
95682e2f by Volker Lendecke at 2018-09-07T15:26:18Z
dbwrap: Remove a pointless "return;"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
b487979f by Martin Schwenke at 2018-09-07T15:26:18Z
ctdb-tests: Fix CTDB -O3 --picky-developer build on CentOS 7

gcc 4.8.5 complains:

[319/381] Compiling ctdb/tests/src/system_socket_test.c
../tests/src/system_socket_test.c: In function ‘test_tcp’:
../tests/src/system_socket_test.c:196:20: error: ‘rst_out’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
  assert((rst != 0) == (rst_out != 0));
                    ^
cc1: all warnings being treated as errors

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
70169d47 by Andreas Schneider at 2018-09-07T18:24:46Z
wafsamba: Fix 'make -j<jobs>'

Currently only 'make -j' enables parallel builds and e.g. 'make -j4'
results in no parallel compile jobs at all.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13606

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Sep  7 20:24:46 CEST 2018 on sn-devel-144

- - - - -
a2aa7d6d by Günther Deschner at 2018-09-07T23:43:27Z
s3-spoolss: Make spoolss client os_major,os_minor and os_build configurable.

Similar to spoolss server options, make the client advertised OS version
values configurable to allow overriding the defaults provided to the print server.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13597

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
1da77084 by Justin Stephenson at 2018-09-07T23:43:27Z
s3-rpc_client: Advertise Windows 7 client info

Client printing operations currently fail against Windows
Server 2016 with Access Denied if a client os build number
lower than 6000 is advertised. Increase the default build number,
major, and minor versions to values associated with client
OS versoins Windows 7 and Windows Server 2008 R2.

The build number value specifically needs to be increased to
allow these operations to succeed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13597

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b8279f65 by Günther Deschner at 2018-09-07T23:43:27Z
s3-rpcclient: Use spoolss_init_spoolss_UserLevel1 in winspool cmds

Use spoolss initialization function to set client version information for
iremotewinspool printer operations

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
4fc0602d by Justin Stephenson at 2018-09-07T23:43:27Z
iremotewinspool-tests: Allow modification of OS client version information

Add test_get_client_info() function to set and, or modify the client OS
version values advertised in the iremotewinspool torture tests.

The OS build numbers are used from the table in:

  [MS-RPRN] <168> Section 2.2.3.10.1

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
2b970741 by Justin Stephenson at 2018-09-07T23:43:28Z
iremotewinspool-tests: Add client os build number validation test

Add test validating the AsyncOpenPrinter result based on the provided
client info build number

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b1bf9c5d by Justin Stephenson at 2018-09-08T02:35:56Z
spoolss-iremotewinspool-tests: Use more recent client OS version

Set torture test client info build, major, and minor
version numbers to Windows 7 and Windows Server 2008 R2 values

 buildnum: 7007
 major: 6
 minor: 1

Build number taken from
 [MS-RPRN] <168> Section 2.2.3.10.1

Major/Minor numbers taken from
 https://docs.microsoft.com/en-us/windows/desktop/sysinfo/operating-system-version

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Sep  8 04:35:56 CEST 2018 on sn-devel-144

- - - - -
95ba6b97 by Alexander Bokovoy at 2018-09-08T06:44:16Z
wafsamba/samba_autoconf: when setting undefined result, use empty tuple

A difference between waf 1.x and 2.x is that we gained 0 as an undefined
variable in the cache file. This does not allow to differentiate unset
and set to 0 defines.

Force to use empty tuple () to signify unset defines.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8ec7eb0c by Alexander Bokovoy at 2018-09-08T06:44:16Z
lib/replace: force to unset defines if check has failed

For HAVE_WORKING_STRPTIME and HAVE_INCOHERENT_MMAP we always want to
have their defines unset as the code is using '#ifdef' rather than just
'#if'. Setting them to 0 means #ifdef will succeed due to a difference
how '#ifdef' and '#if' are working.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8c00c017 by Jeremy Allison at 2018-09-08T09:37:44Z
s4: torture: Ensure raw notify tests use different directories.

Should help track down flapping tests.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Sep  8 11:37:44 CEST 2018 on sn-devel-144

- - - - -
4e5d6f81 by Douglas Bagnall at 2018-09-10T14:51:09Z
selftext: Mark py3 visualize test as flapping

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Mon Sep 10 16:51:09 CEST 2018 on sn-devel-144

- - - - -
200ddf8e by Stefan Metzmacher at 2018-09-10T17:28:14Z
Revert "lib/replace: force to unset defines if check has failed"

This reverts commit 8ec7eb0c0129518557d1f446191860a62ef3ff79.

There was already a better fix under discusion.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
93b23570 by Stefan Metzmacher at 2018-09-10T17:28:14Z
Revert "wafsamba/samba_autoconf: when setting undefined result, use empty tuple"

This reverts commit 95ba6b97441b75f28aef5ec1ee5a9442683f3763.

There was already a better fix under discusion.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
230796d7 by Alexander Bokovoy at 2018-09-10T17:28:15Z
buildtools/wafsamba: undefine a define for a failed test unless asked to keep it

If conf.CHECK_CODE() is called without `always=True` and the test has
failed, undefine the define already set to '0' by conf.check_code().

This restores expectations that undefined symbols are not considered to
be set by CONFIG_SET() method.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0cd9452e by Stefan Metzmacher at 2018-09-10T17:28:15Z
wafsamba: fix generic_cc.py to work with waf 2

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
420d0b56 by Stefan Metzmacher at 2018-09-10T17:28:15Z
wafsamba: remove irixcc.py as we don't support IRIX any more

There's still a irixcc.py in waf itself.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b35ac32b by Stefan Metzmacher at 2018-09-10T17:28:15Z
wafsamba: remove hpuxcc.py as it's not compatible with waf 2

third_party/waf/waflib/Tools/compiler_c.py proposed
gcc for HPUX.

If there's more needed on HPUX someone with a HPUX box needs to
provide a waf 2 compatible fix.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ad042757 by Stefan Metzmacher at 2018-09-10T20:49:31Z
wafsamba: remove tru64cc.py as it's not compatible with waf 2

third_party/waf/waflib/Tools/compiler_c.py proposed
gcc for osf1V/Tru64.

If there's more needed on Tru64 someone with a Tru64 box needs to
provide a waf 2 compatible fix.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Sep 10 22:49:31 CEST 2018 on sn-devel-144

- - - - -
a122428a by Martin Schwenke at 2018-09-11T04:59:11Z
ctdb-build: Use wafsamba's INSTALL_DIR()

install_dir() doesn't work (doesn't respect $DESTDIR) and is only
available for backward compatibilty.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
26ea0f58 by Martin Schwenke at 2018-09-11T04:59:11Z
wafsamba: Make INSTALL_DIR() respect $DESTDIR

INSTALL_DIR() currently ignores $DESTDIR and just installs directories
into the final destination.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
b4fa3f98 by Martin Schwenke at 2018-09-11T04:59:11Z
wafsamba: Have INSTALL_DIR() log directory creation

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
587bbd3e by Martin Schwenke at 2018-09-11T08:00:05Z
wafsamba: Drop unused, broken install_dir()

This is broken because it doesn't respect $DESTDIR.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Alexander Bokovoy <ab at samba.org>
Autobuild-Date(master): Tue Sep 11 10:00:05 CEST 2018 on sn-devel-144

- - - - -
94c6a7d7 by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Remove an unneeded #include

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dd89034f by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Simplify close_directory()

Same patch as in 8541829a9ab20c7fa8c

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5ef3610f by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Remove an unneeded #include

ndr_open_files already includes open_files.h

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f741f4d1 by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Factor out "has_other_nonposix_opens"

This is exactly the same in both file and directory cases

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e84da7ec by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Use has_other_nonposix_opens in smb_posix_unlink

Almost the same code as in close.c. has_other_nonposix_opens() is a bit
more general, but the purpose is the same.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e24f3ff4 by Jeremy Allison at 2018-09-11T16:36:23Z
s4: torture: Fix "flakey" notify test on slow cloud filesystems.

Ensure we keep asking for change notifies until we get them all
(or the request times out).

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
885435e8 by Andreas Schneider at 2018-09-11T19:43:28Z
s3:libsmbclient: Fix a typo

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Michael Adam <obnox at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Sep 11 21:43:28 CEST 2018 on sn-devel-144

- - - - -
204e054b by Jeremy Allison at 2018-09-11T20:45:10Z
s3: smbd: Initialization cleanup.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Michael Adam <obnox at samba.org>

- - - - -
de754105 by Jeremy Allison at 2018-09-11T20:45:10Z
s3: smbd: Restructure check_parent_exists() to ensure all non-optimization returns go to one place.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Michael Adam <obnox at samba.org>

- - - - -
ba9d293c by Jeremy Allison at 2018-09-11T20:45:10Z
s3: smbd: Ensure dirpath is set to ".", not "\0" so it is always a valid path.

Cleanup of the internals of unix_convert().

Ensure check_parent_exists() returns this in the non-optimization
case. Ensure unix_convert() initializes dirpath to ".".

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Michael Adam <obnox at samba.org>

- - - - -
0993141c by Michael Adam at 2018-09-12T00:05:47Z
smbd:vfs: fix mis-spellings of hierarchy in comments

Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Sep 12 02:05:47 CEST 2018 on sn-devel-144

- - - - -
3903f6c3 by Martin Schwenke at 2018-09-12T03:50:46Z
ctdb-build: Fix version handling when building tarball

Split get_version() into 2 functions, so that .distversion file can be
created.

Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Wed Sep 12 05:50:46 CEST 2018 on sn-devel-144

- - - - -
cc30805e by Björn Baumbach at 2018-09-12T16:03:10Z
samba_dnsupdate: honor 'dns zone scavenging' option, only update if needed

Since scavenging is implemented the samba_dnsupdate command always updates all
dns records required by the dc. This is not needed if dns zone scavenging
is not enabled.

This avoids the repeating TSIG error messages:

 # samba_dnsupdate --option='dns zone scavenging = yes' 2>&1 | uniq -c
     29 ; TSIG error with server: tsig verify failure
      1 Failed update of 29 entries
 # echo ${PIPESTATUS[0]}
 29

 # samba_dnsupdate --option='dns zone scavenging = no' 2>&1 | uniq -c
 # echo ${PIPESTATUS[0]}
 0

Note that this results in about 60 lines in the log file,
which triggered every 10 minutes ("dnsupdate:name interval=600" is the default).

This restores the behavior before 8ef42d4dab4dfaf5ad225b33f7748914f14dcd8c,
if "dns zone scavenging" is not switched on (which is still the default).

Avoiding the message from happening at all is subject for more debugging,
most likely they are caused by bugs in 'nsupdate -g' (from the bind package).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13605

Pair-programmed-with: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Björn Baumbach <bb at sernet.de>
Signed-off-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Björn Baumbach <bb at sernet.de>
Autobuild-Date(master): Wed Sep 12 18:03:10 CEST 2018 on sn-devel-144

- - - - -
e52abc8a by Ralph Wuerthner at 2018-09-12T19:50:57Z
ctdb-doc: Remove PIDFILE option from ctdbd_wrapper man page

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13610

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Ralph Böhme <slow at samba.org>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Wed Sep 12 21:50:57 CEST 2018 on sn-devel-144

- - - - -
b1838b15 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap: move sockname variable and call to lp_ctdbd_socket into context

sockname is only needed in a cluster.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6617b2db by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap: early return, removes an indentation level

No change in behaviour.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9a9620bb by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_tdb: move a function call out of an if condition

At least for me this improves readability somewhat. No change in
behaviour.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
382620d4 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_tdb: use struct initializer

This ensures all struct members are implicitly initialized.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7cafbc50 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: add error checking to ctdbd_dbpath()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
daea9655 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: simplify if condition

This just moves the talloc_memdup() out of the if condition as per
README.Coding.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dcc50d67 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: README.Coding fixes in traverse_callback()

NULL initialize pointers, check function return values, explicit
variable check against NULL.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fce54bf0 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: use struct initializer in db_ctdb_traverse_read()

This ensures all struct members are implicitly initialized.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
eaaee00a by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: use struct initializer in db_ctdb_traverse()

This ensures all struct members are implicitly initialized.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7c0607e9 by Ralph Boehme at 2018-09-12T20:25:20Z
dbwrap_ctdb: increment record count in traverse_callback()

state->count wasn't incremented and is returned at the end of a
dbwrap_traverse().

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f84e372a by Ralph Boehme at 2018-09-12T23:34:30Z
dbwrap_ctdb: return correct record count for a persistent db read-only traverse

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Sep 13 01:34:30 CEST 2018 on sn-devel-144

- - - - -
93d72bcf by Martin Schwenke at 2018-09-13T07:37:23Z
wafsamba: Only use $DESTDIR in INSTALL_DIR() if it is set

Otherwise the leading '/' is stripped and directories are created
relative to the current directory.

This fixes a regression introduced in recent commit
26ea0f58daace4adef7c5bb17f19476083bf3b7b.

Reported-by: Ralph Böhme <slow at samba.org>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Thu Sep 13 09:37:23 CEST 2018 on sn-devel-144

- - - - -
5e4f34e7 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Add some structure protection for durable reconnect

We should consume all data, and the ndr_pull function fills in all
fields. Thus the ZERO_STRUCT(cookie) is not required.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
b6b7c157 by Volker Lendecke at 2018-09-14T05:49:13Z
dbwrap_tdb: Avoid double-call to talloc_get_type_abort

We've already retrieved "ctx" in the callers of db_tdb_fetch_locked_internal().

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
551f8546 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Add a paranoia check for leases

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
e4498de1 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Avoid casts in DEBUG statements

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
41f2d1d1 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Avoid casts in DEBUG statements

Some indentation changed, best viewed with "git show -w"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
aa30fd54 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Remove "file_sync_all" function

Replace with a call to files_forall. Why? I just came across this
function that only has one pretty obscure user. This does not justify
a full library function, IMHO at least.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
92b01adf by Volker Lendecke at 2018-09-14T05:49:13Z
tevent: Fix a docu typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
89e500d8 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Fix a false comment

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9e515f7c by Volker Lendecke at 2018-09-14T05:49:14Z
smbd: Simplify share_mode_lock.c

"the_lock_id" is not required here. The share mode data carry the file
id, so use that.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
50adbbe0 by Volker Lendecke at 2018-09-14T05:49:14Z
smbd: Simplify parse_share_modes

Since 823bc4c07ad pidl initializes the [skip] entries

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
36055dbb by Volker Lendecke at 2018-09-14T05:49:14Z
streams_xattr: Make error handling more obvious

Do the NULL check right after the alloc call

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
aca0f2a1 by Volker Lendecke at 2018-09-14T05:49:14Z
streams_xattr: Make error handling more obvious

Do the NULL check right after the alloc call

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
92d3e3a8 by Volker Lendecke at 2018-09-14T05:49:14Z
mdssvc: Simplify sl_unpack_loop

Move a variable declaration closer to its use, avoid a redundant
?true:false;

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
71c70607 by Volker Lendecke at 2018-09-14T05:49:14Z
mdssvc: Move a variable declaration closer to its use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
98cf04c7 by Volker Lendecke at 2018-09-14T05:49:14Z
mdssvc: Move a variable declaration closer to its use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f8df3f34 by Volker Lendecke at 2018-09-14T08:48:47Z
mdssvc: Move a variable declaration closer to its use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Sep 14 10:48:47 CEST 2018 on sn-devel-144

- - - - -
18ab3ea7 by Christof Schmitt at 2018-09-15T13:18:25Z
torture: Remove unnecesssary #include <fcntl.h>

This is now provided through system/filesys.h.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
181e3966 by Noel Power at 2018-09-15T13:18:25Z
python/samba/provision: PY3 port to run samba.tests.upgradeprovisionneeddc

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5517d526 by Noel Power at 2018-09-15T13:18:25Z
s4/selftest: enable samba.tests.upgradeprovisionneeddc for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
12949bb0 by Noel Power at 2018-09-15T13:18:25Z
s4/selftest: enable samba.tests.samba_tool.rodc for PY3

- - - - -
5e21334d by Noel Power at 2018-09-15T13:18:25Z
s4/dsdb: py_dsdb_DsReplicaAttribute should deal with bytes in py3

Seems the underlying c code expects binary blob, so.. we should
handle str for py2 and byte for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6b30bcf7 by Noel Power at 2018-09-15T13:18:26Z
fallout from py_dsdb_DsReplicaAttribute to -> bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cea2fd6c by Noel Power at 2018-09-15T13:18:26Z
s4/param: py2/p3 compat override_prefixmap should be string/bytes

- - - - -
dcbdae15 by Noel Power at 2018-09-15T13:18:26Z
python/samba: ndr_unpack has to take bytes not string

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fc13a126 by Noel Power at 2018-09-15T13:18:26Z
python/samba: PY3 port for ridalloc_exop test to work

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bf2c6029 by Noel Power at 2018-09-15T13:18:26Z
s4/selftest: enable samba4.drs.ridalloc_exop for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
939957fc by Noel Power at 2018-09-15T13:18:26Z
python/samba/netcmd: PY3 fix test samba.tests.samba_tool.passwordsettings

- - - - -
7afa2d93 by Noel Power at 2018-09-15T13:18:26Z
s4/selftest: enable samba.tests.samba_tool.passwordsetting for Py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8db43696 by Noel Power at 2018-09-15T13:18:26Z
python/samba: extra ndr_unpack needs bytes function

- - - - -
6941544a by Noel Power at 2018-09-15T13:18:27Z
s4/selftest/tests.py enable samba.tests.getdcname for py3

- - - - -
ed1fd607 by Noel Power at 2018-09-15T13:18:27Z
s4/selftest: enable samba.tests.net_join_no_spnego for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2f5db304 by Noel Power at 2018-09-15T13:18:27Z
s4/selftest: enable samba.tests.net_join for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3530dad6 by Noel Power at 2018-09-15T13:18:27Z
selftest/knownfail.d: add PY3 entry for samba4.ldap.password_settings

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4deb61d6 by Noel Power at 2018-09-15T13:18:27Z
s4/selftest: enable samba4.ldap.password_settings for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1e23962a by Noel Power at 2018-09-15T13:18:27Z
s4/torture/drs: port samba4.drs.samba_tool_drs for py2/py3 compat

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
12ff5df1 by Noel Power at 2018-09-15T13:18:27Z
s4/selftest: enable samba4.drs.samba_tool_drs for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0728cce0 by Noel Power at 2018-09-15T13:18:27Z
s4/torture/drs: PY3 port for test samba_tool_drs_showrepl

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
111fd459 by Noel Power at 2018-09-15T13:18:28Z
s4/torture/drs/python: make test code run in py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c0b18798 by Noel Power at 2018-09-15T13:18:28Z
s4/torture/drs: PY3 fix error with test_samba_tool_showrepl_json

some versions of json.loads appear to need to be passed string.
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0e20bf3d by Noel Power at 2018-09-15T13:18:28Z
selftest/knownfail.d: PY3 rule for replica_sync

- - - - -
13dc820c by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: enable samba4.drs.samba_tool_drs_showrepl py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
89f5f53f by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: samba4.drs.replica_sync.python enable for python3.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
03deaae2 by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: enable samba.tests.samba_tool.forest for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a9295541 by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: enable samba.tests.samba_tool.schema for py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
72497e1f by Noel Power at 2018-09-15T13:18:28Z
s4/torture/drs: port samba4.drs.delete_object test for py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6db0d67f by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: enable test samba4.drs.delete_object for py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
496fc547 by Noel Power at 2018-09-15T13:18:29Z
s4/torture/drs: py2/py3 compat changes for test samba4.drs.fsmo

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a299ff58 by Noel Power at 2018-09-15T13:18:29Z
s4/selftest: enable samba4.drs.fsmo for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4de32576 by Noel Power at 2018-09-15T13:18:29Z
s4/torture/drs: py2/py3 compat for test samba4.drs.repl_move

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
40638fe7 by Noel Power at 2018-09-15T13:18:29Z
s4/selftest: enable samba4.drs.repl_move for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4e6987e5 by Noel Power at 2018-09-15T13:18:29Z
s4/torture/drs/python: ndr_unpack needs bytes samba4.drs.getnc_exop

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
30be125b by Noel Power at 2018-09-15T13:18:29Z
s4/selftest: enable samba4.drs.getnc_exop for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
050ba20f by Noel Power at 2018-09-15T13:18:29Z
s4/selftest: enable samba4.drs.getnc_unpriv for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
71900d92 by Noel Power at 2018-09-15T13:18:29Z
s4/torture/drs: py2/py3 compat porting for samba4.drs.link_conflicts

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9a5a2e06 by Noel Power at 2018-09-15T13:18:30Z
s4/selftest: enable samba4.drs.link_conflicts for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
50c5c47a by Noel Power at 2018-09-15T13:18:30Z
s4/selftest: enable samba4.drs.repl_schema for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
186b7add by Noel Power at 2018-09-15T13:18:30Z
s4/torture/drs/python: py2/py3 porting needed for samba4.drs.getncchanges

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7943e532 by Noel Power at 2018-09-15T13:18:30Z
selftest/knownfail.d: PY3 entry fro getncchanges

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bcefbbb3 by Noel Power at 2018-09-15T13:18:30Z
s4/selftest: enable samba4.drs.getncchanges for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0ed0c07d by Noel Power at 2018-09-15T13:18:30Z
s4/torture/drs/python: py2/py3 compat changes for samba4.drs.repl_rodc

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3451c5e7 by Noel Power at 2018-09-15T13:18:30Z
s4/selftest: enable samba4.drs.repl_rodc for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0fd8c69e by Noel Power at 2018-09-15T13:18:30Z
python/samba/tests: py_gensec_update takes bytes as param

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2eac2f13 by Noel Power at 2018-09-15T13:18:31Z
s4/selftest: enable samba.tests.join for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f18674db by Noel Power at 2018-09-15T13:18:31Z
s4/selftetst: enable samba4.drs.cracknames for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b2665d56 by Noel Power at 2018-09-15T13:18:31Z
s4/selftest: enable samba.tests.kcc for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5884ef29 by Noel Power at 2018-09-15T13:18:31Z
s4/selftest: enable samba.tests.kcc.kcc_utils for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d5413d5b by Noel Power at 2018-09-15T13:18:31Z
s4/selftest: enable samba.tests.netlogonsvc for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7208d4a4 by Noel Power at 2018-09-15T13:18:31Z
selftest/knownfail: Add PY3 entries for samba.tests.ntlmdisabled

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4038e7bd by Noel Power at 2018-09-15T16:10:42Z
s4/selftest: enable samba.tests.ntlmdisabled for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Sep 15 18:10:42 CEST 2018 on sn-devel-144

- - - - -
3cc284b2 by Noel Power at 2018-09-16T04:16:19Z
PY3: fix some octal literals

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
693834f7 by Noel Power at 2018-09-16T04:16:19Z
PY3: fix iteritems usage, not supported in python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c6884138 by Noel Power at 2018-09-16T04:16:19Z
PY3: dict has no iterkeys method

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d20e85d0 by Noel Power at 2018-09-16T04:16:19Z
PY3: iterkeys -> keys

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b2b0f819 by Noel Power at 2018-09-16T04:16:19Z
PY3: iterkeys -> keys

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
82c84513 by Noel Power at 2018-09-16T04:16:19Z
PY3: dict doesn't have has_key

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0358d57d by Noel Power at 2018-09-16T04:16:19Z
PY3: dict_keys doesn't have a sort method

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
59f69a7c by Noel Power at 2018-09-16T04:16:19Z
py2/py3 enclose map with list

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bb9e945e by Noel Power at 2018-09-16T04:16:19Z
PY3: fix string.rsplit usage (doens't exist in PY3)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9fc67597 by Noel Power at 2018-09-16T04:16:19Z
PY3: enclose filter with list as len on result of filter fails

filter returns an iterator in PY3 (and a list in PY2)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b342e6f0 by Noel Power at 2018-09-16T04:16:19Z
PY3: bytes.maketrans, string.maketrans is a PY2 only function

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a0c8c8c8 by Noel Power at 2018-09-16T04:16:19Z
PY3: string.translate string.upper don't exist in PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
eceb9970 by Noel Power at 2018-09-16T04:16:20Z
PY3: decode output of cmd_output for easier string manip

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
899ef5d1 by Noel Power at 2018-09-16T04:16:20Z
PY3: string.upper not in PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e2c0af6b by Noel Power at 2018-09-16T04:16:20Z
PY3: xrange->range

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0df8ef41 by Noel Power at 2018-09-16T04:16:20Z
PY3: use pyembed_libname for library names (otherwise missing deps)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6c62e22c by Noel Power at 2018-09-16T04:16:20Z
PY3: detect talloc_utils (for version of python running the build)

It seems all combinations have potential to fail here :/ however for
the moment making the assumption
  *  if python2 driving the build you are building against libpython2 (with the option of having python3 as 'extra-python'
  *  if python3 driving the build you are building against libpython3 (with the option of having python3 as 'extra-python' NOTE: this isn't inforced it probably should be !!!! In a system with python2 and python3 installed it seems waf needs PYTHON env variable to decide which libpython to build against. It's also an option that configure should use that too (to figure out which talloc_util to use or build)

However.... right now I just want the thing to build

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dc9117e4 by Noel Power at 2018-09-16T04:16:20Z
PY3: ensure output of subprocess.Popen is decoded

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a0851c5f by Noel Power at 2018-09-16T04:16:20Z
PY3: wrap filter calls with list where list is expected

filter in PY2 returns list in PY3 it returns an iterator

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0cb34132 by Noel Power at 2018-09-16T07:21:59Z
s4/librpc: Fix py2 dependecies leaking into py3 libraries

Caused strange crashes when using python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sun Sep 16 09:21:59 CEST 2018 on sn-devel-144

- - - - -
5a6b1398 by Martin Schwenke at 2018-09-17T20:58:19Z
ctdb-cluster-mutex: Reset SIGTERM handler in cluster mutex child

If SIGTERM is received and the tevent signal handler setup in the
recovery daemon is still enabled then the signal is handled and a
corresponding event is queued.  The child never runs an event loop so
the signal is effectively ignored.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e789d0da by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-cluster-mutex: Block signals around fork

If SIGTERM is received and the tevent signal handler setup in the
recovery daemon is still enabled then the signal is handled and a
corresponding event is queued.  The child never runs an event loop so
the signal is effectively ignored.

Resetting the SIGTERM handler isn't enough.  A signal can arrive
before that.

Block SIGTERM before forking and then immediately unblock it in the
parent.

In the child, unblock SIGTERM after the signal handler is reset.  An
explicit unblock is needed because according to sigprocmask(2) "the
signal mask is preserved across execve(2)".

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
59fc0164 by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Clean up taking of recovery lock

No functional changes, just coding style cleanups and debug message
tweaks.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c516e58c by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Re-check master on failure to take recovery lock

If the master changed while trying to take the lock then fail gracefully.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
af22f03d by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Rename hold_reclock_state to ctdb_recovery_lock_handle

This will be a longer lived structure.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a53b264a by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Use talloc() to allocate recovery lock handle

At the moment this is still local and is freed after the mutex is
successfully taken.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c5221674 by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Store recovery lock handle

... not just cluster mutex handle.

This makes the recovery lock handle long-lived and with allow the
releasing code to cancel an in-progress attempt to take the recovery
lock.

The cluster mutex handle is now allocated off the recovery lock
handle.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a755d060 by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Return early when the recovery lock is not held

This makes upcoming changes simpler.

Update to modern debug macro while touching relevant line.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
b1dc5687 by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Handle cancellation when releasing recovery lock

If the recovery lock is in the process of being taken then free the
cluster mutex handle but leave the recovery lock handle in place.
This allows ctdb_recovery_lock() to fail.

Note that this isn't yet live because rec->recovery_lock_handle is
still only set at the completion of the attempt to take the lock.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
486022ef by Martin Schwenke at 2018-09-18T00:18:30Z
ctdb-recoverd: Set recovery lock handle at start of attempt

This allows the attempt to be cancelled if an election is lost and an
unlock is done before the attempt is completed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Tue Sep 18 02:18:30 CEST 2018 on sn-devel-144

- - - - -
d0d1eafa by Martin Schwenke at 2018-09-18T08:33:57Z
gitlab-ci: Correct the ctdb tasks

They're swapped.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Tue Sep 18 10:33:57 CEST 2018 on sn-devel-144

- - - - -
8453c617 by Christof Schmitt at 2018-09-18T22:54:23Z
vfs_gpfs: Check for GPFS file system on connect

The vfs_gpfs modules uses GPFS API calls that only succeed when using
the module with the GPFS file system. Add an explicit statfs check for
the file system type on connect, to make it obvious when the file system
is missing or not mounted. The check can be skipped by setting
gpfs:check_fstype to 'no'.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
8b4ec1c2 by Christof Schmitt at 2018-09-19T01:56:47Z
docs: Add gpfs:check_fstype to vfs_gpfs manpage

Signed-off-by; Christof Schmit <cs at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Wed Sep 19 03:56:47 CEST 2018 on sn-devel-144

- - - - -
e5298c8b by Noel Power at 2018-09-19T15:49:33Z
uid_wrapper: Fix Tab/indent error with python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Wed Sep 19 17:49:33 CEST 2018 on sn-devel-144

- - - - -
48ad1db3 by Richard Sharpe at 2018-09-19T15:52:13Z
s3: Don't copy back the stat struct from stat_cache_lookup if we did not get a hit on the full path.

Signed-off-by: Richard Sharpe <realrichardsharpe at gmail.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c3211c8e by Anoop C S at 2018-09-19T20:23:28Z
s3/locking: Fix comments to reflect code flow

* posix_pending_close_db is an in-memory database and not a tdb.
  Therefore adjusting comments to convey the correct meaning of the
  database.
* we do not have posix_locking_close_file() any more which got renamed
  to locking_close_file(). Thus fixing comment to mention the new name.

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Sep 19 22:23:28 CEST 2018 on sn-devel-144

- - - - -
2e59a334 by Noel Power at 2018-09-19T20:25:05Z
PY3: make sure print stmt is enclosed by '(' & ')'

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1d1cd28a by Andrew Bartlett at 2018-09-19T20:25:05Z
s3: nmbd: Stop nmbd network announce storm.

Correct fix for. On announce, work->lastannounce_time is set
to current time t, so we must check that 't >= work->lastannounce_time',
not 't > work->lastannounce_time' otherwise we end up not
doing the comparison, and always doing the announce.

Reported by Reuben Farrelly

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13620

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Revviewe-by: Andreas Schneider <asn at samba.org>

- - - - -
cf7e7710 by Andrew Bartlett at 2018-09-19T23:29:47Z
Fix pdb_samba_dsdb build on Ubuntu 16.04 with -O3 -Werror=strict-overflow

../source3/passdb/pdb_samba_dsdb.c: In function ‘pdb_samba_dsdb_set_trusteddom_pw’:
../source3/passdb/pdb_samba_dsdb.c:2778:6: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow]
   if (i == 0) {
      ^
cc1: all warnings being treated as errors

gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Sep 20 01:29:48 CEST 2018 on sn-devel-144

- - - - -
b6efa150 by Amitay Isaacs at 2018-09-21T01:06:44Z
heimdal-build: Avoid using python str.format()

In python 2.6, the format fields need to be numbered explicitly.  This
causes the build to fail on RHEL6/Centos6 with following error:

 File "/home/amitay/samba.git/source4/heimdal_build/wscript_build", line 87, in HEIMDAL_ASN1
    os.path.join(bld.path.abspath(), option_file)))
ValueError: zero length field name in format

To use str.format() in HEIMDAL_ASN1(),

   "--option-file='{}'"  needs to be   "--options-file='{0}'"

Or this fix which avoids str.format() completely.

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Fri Sep 21 03:06:44 CEST 2018 on sn-devel-144

- - - - -
3b94d31d by Amitay Isaacs at 2018-09-21T18:04:22Z
ldb: Fix dependency of ldb module

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13626
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
198bcfba by Douglas Bagnall at 2018-09-21T18:04:23Z
python tests: fix format() strings for Python 2.6

Python 2.6 wants "{0}".format(x), not "{}".format(x).

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
93bbe2d2 by Tim Beale at 2018-09-21T18:04:23Z
tests: Minor code cleanups to confidential_attr test

+ fix a couple of flake8 warnings
+ add some extra code comments (particularly around the cases where the
  child class overrides a particular method, to avoid confusion when
  browsing the code).
+ assert_not_in_result() was duplicated (it's only needed for the deny
  ACL tests)
+ skip redundant if in dirsync's assert_search_result() (it always has
  to use the base-DN - we never pass it this as an args).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6c1ff590 by Tim Beale at 2018-09-21T18:04:23Z
acl_read: Rework Samba code to reflect Windows logic

This patch should not alter functionality. It is just updating the Samba
code to better match the Windows specification docs.

When fixing Samba BUG #13434, the Microsoft behaviour wasn't clearly
documented, so we made a best guess based on observed behaviour.
The problem was an exception was made to allow "objectClass=*" searches
to return objects, even if you didn't have Read Property rights for the
object's objectClass attribute. However, the logic behind what
attributes were and weren't covered by this exception wasn't clear.

I made a guess that it was attributes belonging to the Public Info
property-set that also have the systemOnly flag set.

Microsoft have confirmed the object visibility behaviour. It turns out
that an optimization is made for the 4 attributes that are always
present for every object (i.e. objectClass, distinguishedName,
name, objectGUID). They're updating their Docs to reflect this.

Now that we know the Windows logic, we can update the Samba code.
This simplifies the code somewhat.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5d866751 by Tim Beale at 2018-09-21T18:04:23Z
tests: test ldap search requesting non-existent attribute

As an after-thought to commit 563e454e8c55e94a950, we thought it
might be a good idea to add a test case that requests an non-existent
attribute in the attribute-filter as well the search-filter.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
18854107 by Tim Beale at 2018-09-21T20:58:17Z
selftest: Tweak PSO test-suite name

There are 2 different PSO tests:
- make test TESTS=ldap.password_settings
- make test TESTS=samba_tool.passwordsettings

There's also another test that's completely unrelated to PSOs:
- make test TESTS=blackbox.password_settings

This patch renames ldap.password_settings --> ldap.passwordsettings.
This means 'make test TESTS=passwordsettings' will run both PSO tests,
but not the unrelated blackbox test.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Sep 21 22:58:17 CEST 2018 on sn-devel-144

- - - - -
d50aafe9 by Philipp Gesang at 2018-09-22T04:05:06Z
lib/audit_logging: make json_{is_invalid,to_string}() accept a const*

Allow for json_is_invalid() and json_to_string() to be used on a
const pointer. Neither function requires for the json object to
be mutable so constraining them to non-const* is unnecessary.

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2e00ad44 by Philipp Gesang at 2018-09-22T04:05:06Z
s3: net: implement json output for ads info

Add the switch '--json' to 'net' to format the output as JSON.

The rationale is to supply the information in a machine-readable
fashion to complement the text version of the output which is
neither particularly well defined nor locale-safe.

The output differs from that of plain 'info' in that times are
not formatted as timestamps.

Currently affects only the 'net ads info' subcommand.

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5534b924 by Philipp Gesang at 2018-09-22T04:05:06Z
s3: net: implement json output for ads lookup

Add JSON printer (option '--json') for the 'net ads lookup'
command. This outputs the same information as the plain version,
with integral ({LMNT,LM20} Token, NT Version) and boolean values
(Flags) not stringified.

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2a19b4e3 by Philipp Gesang at 2018-09-22T04:05:07Z
s3: net: normalize output of lookup subcommand

Use spaces and tabs consistently following the majority of the
printed output: tabs only for indenting, no space before the
colon separator, a single space after the separator.

The irregularities in formatting date back to the original commit
2c029a8b96..

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6dc9f70b by Philipp Gesang at 2018-09-22T07:20:09Z
tests/blackbox: add test for net ads JSON output

Implement blackbox tests for

    $ net ads info --json
    $ net ads lookup --json

that validate

    a) JSON wellformedness (by feeding it into the JSON library
       that ships with Python), and
    b) equality of the set of keys printed to that of the
       non-JSON version.

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Sep 22 09:20:09 CEST 2018 on sn-devel-144

- - - - -
08a5fbd8 by Martin Schwenke at 2018-09-24T05:03:09Z
s3: Fix the build when compiling without JSON support

[3762/3895] Compiling source3/utils/net_ads_gpo.c
../source3/utils/net_ads.c: In function ‘net_ads_cldap_netlogon_json’:
../source3/utils/net_ads.c:311:2: error: parameter name omitted
  (ADS_STRUCT *, const char *,
  ^
../source3/utils/net_ads.c:311:2: error: parameter name omitted
../source3/utils/net_ads.c:312:16: error: parameter name omitted
   const struct NETLOGON_SAM_LOGON_RESPONSE_EX *)
                ^
../source3/utils/net_ads.c: In function ‘net_ads_info_json’:
../source3/utils/net_ads.c:520:1: error: parameter name omitted
 static int net_ads_info_json(ADS_STRUCT *)
 ^

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Mon Sep 24 07:03:09 CEST 2018 on sn-devel-144

- - - - -
2055b710 by Tim Beale at 2018-09-26T05:49:17Z
netcmd: Tweak backup-offline output to avoid subunit truncation

Currently a backup-offline test is occasionally flapping in autobuild,
however, the output is truncated so we can't see what the actual problem
is. The output only ever contains the list of backup dirs. I suspect
that the ']' character printed at the end of the python list might be
getting interpretted by subunit as the end of *all* the output.

If so, we should be able to avoid the problem by printing the list items
without the '['/']'s, i.e. join the list into a single string.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e1f255a4 by Tim Beale at 2018-09-26T05:49:17Z
netcmd: Add --site option when restoring a domain

Restoring a backup only worked if the Default-First-Site-Name site was
still present. When the new restored DC account is created, it was
trying to add the new server's DN under CN=Default-First-Site-Name.
However, if the original domain was setup using a different site, then
the restore would fail because the DN didn't exist.

When running the restore command, you should be able to specify the
site that you want the new/restored DC to be in (same as during a
DC 'join'). Passing the correct --site argument is one way to avoid
this problem. (A subsequent patch will further improve the tool so it
can work around non-default sites automatically).

Note we also need to pass the site through to where the new DNS entries
get registered (in the rename case).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ad69aaf7 by Tim Beale at 2018-09-26T05:49:17Z
tests: Add test-case for restore into non-default site

Add a test-case that exercises the new '--site' restore option and
ensures the restored DC gets added to the correct site.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ce57a800 by Tim Beale at 2018-09-26T05:49:17Z
netcmd: Re-create default site for backup-restore (if missing)

Normally when a new DC joins a domain, samba-tool works out the new
DC's site automatically. However, it does this by querying the existing
DC using CLDAP. In the restore case, there is no DC running. We could
still query the DB on disk and work out the correct site based on the
new DC's IP, however:
- comparing between the CN=Subnet DNs and an IP-address string seems
  like it'd be non-trivial to write, and
- in the lab-domain rename case, chances are the user will want a
  completely different subnet to what's already in the DB.

The restore command now has a --site option so the user can specify an
appropriate site for the restored DC. This patch makes the restore
command work by default (i.e. without a --site option) even if the
default Default-First-Site-Name doesn't exist. Basically the solution is
to just check Default-First-Site-Name exists and create it if it
doesn't. As the recommended workflow is to use the restored DC as a
temporary seed that you'll later throw away, this approach seems
acceptable. Subsequent DCs will then be joined to the running restored
DC, so an appropriate site will be determined using CLDAP. The only
side-effect is potentially an extra Site object.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
696fa6a1 by Tim Beale at 2018-09-26T05:49:17Z
selftest: Change backup testenvs to use non-default site

Previously (i.e. up until the last patch) the backup/restore commands
only worked if the Default-First-Site-Name site was present. If this
site didn't exist, then the various restore testenvs would fail to
start. This is now fixed, but this patch changes the backupfrom testenv
so that it uses a non-default site. This will detect the problem if it
is ever re-introduced.

To do this we need to change provision_ad_dc() so the
extra_provision_options can be specified as an argument. (Note that Perl
treats undef the same as an empty array).

By default, the restore will add the new DC into the
Default-First-Site-Name site. This means the backupfromdc and restored
testenvs will now have different sites, so we need to update the ldapcmp
filters to exclude site-specific attributes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
89543af0 by Tim Beale at 2018-09-26T05:49:17Z
selftest: Test join against DC with non-default site

Recent changes around restoring a domain that lacked
Default-First-Site-Name highlighted a problem. Normally when you join a
DC to a domain, samba-tool works out the correct site to use
automatically. However, if the join uses '--server' to select a DC, then
this doesn't work. It defaults back to Default-First-Site-Name, and the
join command fails if this site doesn't exist.

All the testenvs had Default-First-Site-Name present, so this was never
tested. Now the backupfromdc no longer has a Default-First-Site-Name
site, so running a simple join against that DC fails, highlighting the
problem.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9e81d43a by Tim Beale at 2018-09-26T05:49:17Z
join: Avoid duplicating "Default-First-Site-Name" string

The provision code already defines "Default-First-Site-Name" so we might
as well reuse it.

The join.py already uses a suitable default, so assigning the default in
the domain netcmd code is unnecessary.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
26dd30d6 by Tim Beale at 2018-09-26T05:49:18Z
join: Support site detection when --server is specified

When a new DC is joined to the domain, samba-tool would automatically
detect an appropriate site for the new DC. However, it only did this if
the --server option wasn't specified. The new DC's site got
automatically updated as part of the finddc() work, however, this step
gets skipped if we already know the server DC to join to.

In other words, if Default-First-Site-Name doesn't exist and you specify
--server in the join, then you have to also specify --site manually,
otherwise the command fails. This is precisely what's happening in the
join_ldapcmp.sh test, now that the backupfromdc testenv no longer has the
Default-First-Site-Name present.

This patch adds a new find_dc_site() function which uses the same
net.finddc() API (except based on the server-address rather than
domain-name). Assigning DEFAULTSITE has been moved so that it only
gets done if finddc() can't determine the site.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4e592c91 by Douglas Bagnall at 2018-09-26T05:49:18Z
domain_backup test: fix py2.6 incompatible format

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
36cb85be by Douglas Bagnall at 2018-09-26T08:40:49Z
samba-tool domain backup: fix py2.6 incompatible format

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep 26 10:40:49 CEST 2018 on sn-devel-144

- - - - -
cb8e61b5 by Andreas Schneider at 2018-09-26T18:10:07Z
docs: Only build vfs manpages if the module is enabled

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

Autobuild-User(master): Günther Deschner <gd at samba.org>
Autobuild-Date(master): Wed Sep 26 20:10:07 CEST 2018 on sn-devel-144

- - - - -
c98f9971 by Philipp Gesang at 2018-09-26T18:41:07Z
turn --with-json-audit into global --with-json

Fold the build option --with-json-audit into the toplevel wscript
to reflect the fact that JSON support is no longer local to the
audit subsystem.

Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5a73f904 by Douglas Bagnall at 2018-09-26T21:40:58Z
py3 kcc tests: mark verify test as flapping

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep 26 23:40:58 CEST 2018 on sn-devel-144

- - - - -
d786e1fc by Noel Power at 2018-09-26T23:54:26Z
lib/ldb: Test correct variable for no mem condition

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d1492ab9 by Noel Power at 2018-09-26T23:54:26Z
lib/ldb/tests: add test for ldb.Dn passed utf8 unicode

object dn format should be a utf8 encoded string
Note: Currently this fails in python2 as the c python binding for
      the dn string param uses PyArg_ParseTupleAndKeywords() with 's'
      format, this will accept str *or* unicode in the default encoding.
      The default encoding in python2 is... ascii.

Also adding here a knownfail to squash the error produced by the test.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cddd54e8 by Noel Power at 2018-09-26T23:54:26Z
lib/ldb: Ensure ldb.Dn can accept utf8 encoded unicode

Additionally remove the associated known fail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13616
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e8fec948 by Noel Power at 2018-09-26T23:54:27Z
PY3: fix "TabError: inconsistent use of tabs and spaces"

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2d94839f by Noel Power at 2018-09-26T23:54:27Z
PY3: decode bytes in py3 where strings are needed

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d28b4fa3 by Noel Power at 2018-09-26T23:54:27Z
PY3: md5 related functions need to be passed bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f71675dc by Noel Power at 2018-09-26T23:54:27Z
python/samba: PY3 add compat function urllib_join to replace urllib.urljoin

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2ea47f3e by Noel Power at 2018-09-26T23:54:27Z
s4/web_server: PY3: port to python3

Note: Unlike other libraries this library is been only built for the
configured python version. It depends on availability of 'swat' python
module.

The swat module is hosted externally, it seems not to have been modified
for a number of years, I don't think swat is python3 compatabile. These
changes are enough to get allow the samba binary to launch a web server
(which will just display a placeholder page announcing you need to install
swat). It maybe that removing this functionality is what we should do, but
that is a decision that can be made at a later time.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9a6052b5 by Noel Power at 2018-09-26T23:54:27Z
buildtools/wafsamba: fix basestring not defined error in PY3

Test for str first (which exists in py3 & py2) this avoids
the undefined runtime error.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8514b5b1 by Noel Power at 2018-09-26T23:54:27Z
buildtools/wafsamba: add_manual_dependency needs bytes for value

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1c16c0cf by Aaron Haslett at 2018-09-26T23:54:27Z
netcmd: domain backup offline bug fix - ignore sock files

Ignoring autogenerated .sock files and directories during file system based
offline domain backup.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13604
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8c754b43 by Aaron Haslett at 2018-09-27T02:47:16Z
dlz_bind9: torture tests exploring rndc reload behaviour

These tests establish that the process triggered by the command 'rndc reload'
does not cause samba's bind9 dlz plugin to crash.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13214

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Sep 27 04:47:16 CEST 2018 on sn-devel-144

- - - - -
6400c8c6 by Noel Power at 2018-09-27T23:01:23Z
buildtools/wafsamba: Finally fix reference to basestring PY3 error.

While a previous attempt squashed the error on the config & make
phase, make install threw up this error again.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
47634a25 by Noel Power at 2018-09-27T23:01:23Z
PY3: We support python3 now, remove error when python3 detected

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
de1e6d78 by Noel Power at 2018-09-27T23:01:23Z
script: add new autobuild task for building pure python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f08ba3a4 by Noel Power at 2018-09-27T23:01:23Z
add new gitlab CI job for building pure python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9c0a9c69 by Noel Power at 2018-09-28T01:57:47Z
s4/web_server: fix panic from wrong type extracted from conn

enabling
  server services = +web

and we get a panic

web_server.c:251: Type mismatch: name[struct web_server_data] expected[struct task_server]
smb_panic_default: PANIC (pid 29137): ../source4/web_server/web_server.c:251: Type mismatch: name[struct web_server_data] expected[struct task_server]

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Sep 28 03:57:47 CEST 2018 on sn-devel-144

- - - - -
0122f45f by Tim Beale at 2018-09-28T06:30:22Z
netcmd: Make sure SMB connection is signed when backing up sysvol

i.e. protect the client against man-in-the-middle attacks by default.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
37870626 by Tim Beale at 2018-09-28T06:30:22Z
s3/smbd: Server responds incorrectly if no SMB protocol chosen

The SMBnegprot response from the server contains the DialectIndex of the
selected protocol from the client's request message. Currently, if no
protocol is selected, the server is responding with a DialectIndex=zero,
which is a valid index (PROTOCOL_CORE by default). The Windows spec, and
historically the code, should return DialectIndex=0xffff if no protocol
is chosen. The following commit changed it recently (presumably
inadvertently), so that it now returns DialectIndex=zero.

06940155f315529c5b5 s3:smbd: Fix size types in reply_negprot()

This results in somewhat confusing error messages on the client side:
ERROR(runtime): uncaught exception - (3221225997, 'The transport
connection has been reset.')

or, when signing is configured as mandatory:
smbXcli_negprot: SMB signing is mandatory and the selected protocol
level (1) doesn't support it.
ERROR(runtime): uncaught exception - (3221225506, '{Access Denied} A
process has requested access to an object but has not been granted those
access rights.')

This patch restores the old behaviour of returning 0xffff.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
34cbd89f by Tim Beale at 2018-09-28T09:25:29Z
libcli: Add debug message if fail to negoatiate SMB protocol

Currently if the client and server can't negotiate an SMB protocol, you
just get the followiing error on the client-side, which doesn't tell you
much.
ERROR(runtime): uncaught exception - (3221225667, 'The network responded
incorrectly.')

This patch adds a debug message to help highlight what's actually going
wrong.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Sep 28 11:25:29 CEST 2018 on sn-devel-144

- - - - -
660dbfae by Jeremy Allison at 2018-09-29T03:32:41Z
s3: smbd: Prevent valgrind errors in smbtorture3 POSIX test.

Missing fsp talloc free and linked list delete in error
paths in close_directory(). Now matches close_normal_file()
and close_fake_file().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13633

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Sep 29 05:32:41 CEST 2018 on sn-devel-144

- - - - -
46e171e2 by Volker Lendecke at 2018-10-02T08:40:05Z
libsmb: Remove smb_share_modes.[ch]

This was declared nonfunctional in 2014. Finally remove it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
380066d2 by Anoop C S at 2018-10-02T08:40:05Z
s3/locking: Fix logging of lock reference count

lock reference count is always increased and reduced by a value of 1.
But lock_ref_count variable holds the old value prior to change and
was being logged wrongly under debug level 10. DEBUG statement must
log lock_ref_count+1 and lock_ref_count-1 respectively when value
gets increased and decreased.

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
01b86845 by Stefan Metzmacher at 2018-10-02T08:40:06Z
s4:torture: split smb2.session.expire{1,2} to run with signing and encryptpion

This reproduces the problem we have with expired encrypted sessions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13624

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4ef45e53 by Stefan Metzmacher at 2018-10-02T12:11:30Z
smb2_server: set req->do_encryption = true earlier

The STATUS_SESSION_EXPIRED error was returned unencrypted,
if the request was encrypted.

If clients use SMB3 encryption and the kerberos authenticated session
expires, clients disconnect the connection instead of doing a reauthentication.

>From https://blogs.msdn.microsoft.com/openspecification/2012/10/05/encryption-in-smb-3-0-a-protocol-perspective/

  The sender encrypts the message if any of the following conditions is
  satisfied:

    - If the sender is sending a response to an encrypted request.
    - If Session.EncryptData is TRUE and the request or response being
      sent is not NEGOTIATE.
    - If Session.EncryptData is FALSE, the request or response being sent
      is not NEGOTIATE or SESSION_SETUP or TREE_CONNECT, and
      <TreeConnect|Share>.EncryptData is TRUE.

[MS-SMB2] 3.3.4.1.4 Encrypting the Message

 If Connection.Dialect belongs to the SMB 3.x dialect family and
 Connection.ClientCapabilities includes the SMB2_GLOBAL_CAP_ENCRYPTION
 bit, the server MUST encrypt the message before sending, if any of the
 following conditions are satisfied:

 - If the message being sent is any response to a client request for which
   Request.IsEncrypted is TRUE.

 - If Session.EncryptData is TRUE and the response being sent is not
   SMB2_NEGOTIATE or SMB2 SESSION_SETUP.

 - If Session.EncryptData is FALSE, the response being sent is not
   SMB2_NEGOTIATE or SMB2 SESSION_SETUP or SMB2 TREE_CONNECT, and
   Share.EncryptData for the share associated with the TreeId in the SMB2
   header of the response is TRUE.

 The server MUST encrypt the message as specified in section 3.1.4.3,
 before sending it to the client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13624

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Oct  2 14:11:30 CEST 2018 on sn-devel-144

- - - - -
20302506 by Stefan Metzmacher at 2018-10-02T12:12:13Z
python/tests: turn GraphError into failure inside of test_verify()

That test was already marked as flapping in commit
5a73f904e192c44e304850287ac439d0b52f7be5.

However the test generates an UNEXPECTED(error)
instead of an UNEXPECTED(failure).

  [67(594)/75 at 36m28s] samba.tests.kcc.python3(vampire_dc)
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  UNEXPECTED(error): samba.tests.kcc.python3.samba.tests.kcc.KCCTests.test_verify(vampire_dc)
  REASON: Exception: Exception: Traceback (most recent call last):
    File "bin/python/samba/tests/kcc/__init__.py", line 80, in test_verify
      attempt_live_connections=False)
    File "bin/python/samba/kcc/__init__.py", line 2659, in run
      ('connected',))
    File "bin/python/samba/kcc/__init__.py", line 2513, in plot_all_connections
      vertex_colors=vertex_colours)
    File "bin/python/samba/kcc/graph_utils.py", line 334, in verify_and_dot
      for p, e, doc in errors)))
  samba.kcc.graph_utils.GraphError: The 'dsa_final CN=NTDS Settings,CN=LOCALVAMPIREDC,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=samba,DC=example,DC=com' graph lacks the following properties:
  connected: the graph is not connected, as the following vertices are unreachable:

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
1fd2a79a by Andreas Schneider at 2018-10-02T12:12:13Z
third_party: Update pam_wrapper to version 1.0.7

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
7dd388a1 by Mathieu Parent at 2018-10-02T12:12:13Z
third_party: Add pam_set_items.so from pam_wrapper

Pair-Programmed-With: Andreas Schneider <asn at samba.org>
Signed-off-by: Mathieu Parent <math.parent at gmail.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
ad5debcb by Mathieu Parent at 2018-10-02T12:12:13Z
nsswitch: Add try_authtok option to pam_winbind

Same as the use_authtok option, except that if the new password is not
valid, PAM will prompt for a password.

Bug-Debian: https://bugs.debian.org/858923
Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/570944

Signed-off-by: Mathieu Parent <math.parent at gmail.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
62400224 by Mathieu Parent at 2018-10-02T15:30:29Z
tests: Check pam_winbind pw change with different options

Pair-Programmed-With: Andreas Schneider <asn at samba.org>

Signed-off-by: Mathieu Parent <math.parent at gmail.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Tue Oct  2 17:30:29 CEST 2018 on sn-devel-144

- - - - -
0e7c5464 by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Move downgrade_share_lease() to smbd/oplock.c

This function is pretty closely entangled with its only caller. In
particular the NT_STATUS_OPLOCK_BREAK_IN_PROGRESS triggers acitivity
in the caller, and that's the only case where "*_l" is being set to
non-NULL. Prepare for cleanup

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
4980e60d by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Use find_share_mode_lease() in downgrade_share_lease

Simple simplification: In locking/ we did not have the direct
reference to find_share_mode_lock.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
664808af by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Slightly simplify downgrade_lease()

As much as I dislike }else{ and prefer early returns, I even more
dislike asking for the same condition in two different ways.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
56139b8e by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Simplify downgrade_lease

To me, the "additive" SMB2_LEASE_WRITE|SMB2_LEASE_HANDLE is easier to
read than the negated ~SMB2_LEASE_READ.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
a93aa151 by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Move downgrade_share_lease into downgrade_lease

The next step will simplify the logic of the code.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
cff1b31c by Volker Lendecke at 2018-10-02T20:22:37Z
smbd: Simplify downgrade_share_lease

Coalesce the NT_STATUS_OPLOCK_BREAK_IN_PROGRESS case into just one
if-condition

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Tue Oct  2 22:22:37 CEST 2018 on sn-devel-144

- - - - -
1853fd4f by Andreas Schneider at 2018-10-02T23:07:15Z
third_party: Update cmocka to version 1.1.3

* Added function to filter tests (cmocka_set_test_filter)
* Fixed fixture error reporting
* Some improvement for API documentation -> https://api.cmocka.org/
* Fixed subunit output on failures
* Do not abort if a test is skipped

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
965d7e90 by Volker Lendecke at 2018-10-02T23:07:15Z
includes: Remove an unused #define

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f8fab07c by Volker Lendecke at 2018-10-02T23:07:15Z
includes: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4060e263 by Volker Lendecke at 2018-10-02T23:07:15Z
includes: Remove an unused #define

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
826a2127 by Volker Lendecke at 2018-10-02T23:07:15Z
lib: Remove unused tdb_pack_append()

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e22d5e5b by Volker Lendecke at 2018-10-02T23:07:15Z
smbd: Fix a warning

gcc complains that the "const" is ignored on function return
types. Right now I'm compiling this file a lot, so silence this
warning :-)

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
43a0e635 by Volker Lendecke at 2018-10-03T02:11:59Z
leases: Streamline leases_db_key a bit

We don't need to talloc the blob, it's always the same size

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Oct  3 04:11:59 CEST 2018 on sn-devel-144

- - - - -
54207bb3 by Alexander Bokovoy at 2018-10-05T11:08:25Z
s4/auth/tests: Fix kerberos test string size

>>> len("user0 at samba.example.com")
23

But the string definition does not take a final '\0' into account.
As per Volker's suggestion, use compiler's support to allocate
the string properly.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
6784ff24 by Martin Schwenke at 2018-10-05T14:16:29Z
ctdbd_conn: Generalise inaccurate error message

Seeing:

  ctdb_read_packet failed: Cannot allocate memory
  [...,  0] ../source3/lib/ctdbd_conn.c:121(cluster_fatal)
  cluster fatal event: ctdbd died

The error is due to a memory allocation failure rather than ctdbd
dying.  However, the error message makes people wonder why ctdbd died.

Another alternative would be to wrap cluster_fatal() and have the
wrapper interpret the return value from ctdb_read_packet() to choose
from a set of more precise messages to pass to cluster_fatal().  For a
memory allocation it isn't strictly necessary to call cluster_fatal(),
but all is probably lost and it is still probably better to try to
exit cleanly as soon as possible instead of crashing somewhere.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Oct  5 16:16:29 CEST 2018 on sn-devel-144

- - - - -
71896fdd by Amitay Isaacs at 2018-10-08T00:46:20Z
ctdb-daemon: Add invalid_records flag to ctdb_db_context

If a node becomes INACTIVE, then all the records in volatile databases
are invalidated.  This avoids the need to include records from such
nodes during subsequent recovery after the node comes out INACTIVE state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
040401ca by Amitay Isaacs at 2018-10-08T00:46:20Z
ctdb-daemon: Don't pull any records if records are invalidated

This avoids unnecessary work during recovery to pull records from nodes
that were INACTIVE just before the recovery.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c4ec99b1 by Amitay Isaacs at 2018-10-08T00:46:20Z
ctdb-daemon: Invalidate records if a node becomes INACTIVE

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
dcc99359 by Martin Schwenke at 2018-10-08T00:46:20Z
ctdb-tests: Add recovery record resurrection test for volatile databases

Ensure that deleted records and vacuumed records are not resurrected
from recently inactive nodes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
202b9027 by Amitay Isaacs at 2018-10-08T00:46:20Z
ctdb-vacuum: Simplify the deletion of vacuumed records

The 3-phase deletion of vacuumed records was introduced to overcome
the problem of record(s) resurrection during recovery.  This problem
is now handled by avoiding the records from recently INACTIVE nodes in
the recovery process.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
ef052397 by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-vacuum: Fix the incorrect counting of remote errors

If a node fails to delete a record in TRY_DELETE_RECORDS control during
vacuuming, then it's possible that other nodes also may fail to delete a
record.  So instead of deleting the record from RB tree on first failure,
keep track of the remote failures.

Update delete_list.remote_error and delete_list.left statistics only
once per record during the delete_record_traverse.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
e15cdc65 by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-vacuum: Remove unnecessary check for zero records in delete list

Since no records are deleted from RB tree during step 1, there is no
need for the check.  Run step 2 unconditionally.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
d18385ea by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-daemon: Drop implementation of RECEIVE_RECORDS control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
81dae71f by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-protocol: Mark RECEIVE_RECORDS control obsolete

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
2f89bd96 by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-protocol: Drop marshalling code for RECEIVE_RECORDS control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
83b3c567 by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-tests: Drop code for RECEIVE_RECORDS control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
a66a9693 by Martin Schwenke at 2018-10-08T00:46:21Z
ctdb-tests: Drop ps_ctdbd()

This was used for debugging tests by ensuring that the arguments to
ctdbd were as expected.  It no longer outputs anything useful because
ctdbd is now started without arguments.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f57e5bbd by Martin Schwenke at 2018-10-08T00:46:21Z
ctdb-tests: Rename ctdb_start_all() -> ctdb_init()

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4642a347 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Rename _ctdb_start_all() -> ctdb_start_all()

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f1ede41a by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Don't used daemons_start()/daemons_stop() directly in tests

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
90f6b0a1 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Drop functions daemons_start(), daemons_stop()

There are too many functions to start/stop daemons.  Simplify this.

Inline the functionality into ctdb_start_all() and ctdb_stop_all().

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2cd6a003 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Explicitly check for local daemons when shutting down

This is clearer if the logic is explicit...  and...

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f2e4a5e9 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Drop unused function maybe_stop_ctdb()

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8bde6fa0 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Don't remove non-existent test database directory

This directory is no longer used.  Lack of removal doesn't seem to
cause a problem.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a9ac3301 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Do not use ctdbd_wrapper in local daemon tests

Run the daemon directly and shut it down using ctdb shutdown.

The wrapper waits for ctdbd to reach >=FIRST_RECOVERY runstate within
a timeout period and shuts ctdbd down if that doesn't happen.  This is
only really used to ensure that ctdbd doesn't exit early after an
apparently successful start.  There are no known cases where ctdbd
will continue running but fail to reach >=FIRST_RECOVERY runstate.

When ctdbd is started in tests, the test code will wait until ctdbd is
in a healthy state on all nodes before proceeding, so there is
effectively no change in behaviour.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0eabac52 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Be more efficient about starting/stopping local daemons

Don't loop, just use onnode all.

For shutting down, use onnode -p all.  This results in a significant
time saving for stopping many deamons because "ctdb shutdown" is now
synchronous.

onnode -p all can be used to start daemons directly because they
daemonize.  However, this does not work under valgrind because the
valgrind process does not exit, so onnode will wait forever for it.
In this case, use onnode without the -p option.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
03dddc37 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Don't format IPv4 octets as hex digits

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
36eb7388 by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Be more careful when building node addresses

The goal is to allow more local daemons by expanding the address range
rather than generating invalid addresses.

For IPv6, use all 4 trailing hex digits.

For IPv4, use the last 2 octets.  Although 127.0.0.0 is a /8 network,
avoid unexpected issues due to 0 and 255 in the last octet.  Use a
maximum of 100 addresses per "subnet" starting at .1.  Keep the first
group of addresses in 127.0.0.0/24 to continue to allow a reasonable
number of nodes to be tested with socket-wrapper.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0dfb3c87 by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Be more careful when building public IP addresses

The goal is to allow more local daemons by expanding the address range
rather than generating invalid addresses.

For IPv6, use a separate address space instead of an offset for the
2nd address.

For IPv4, use the last 2 octets with addresses starting at
192.168.100.1 and 192.168.200.1.  Avoid addresses with 0 and 255 in
the last octet by using a maximum of 100 addresses per "subnet"
starting at .1.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
6ac5124b by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Support closing of stdin in local daemons ssh stub

Not sure this is needed but this makes it behave the same as ssh.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2aa006a3 by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tools: Have onnode pass -n option even when regular ssh not in use

ONNODE_SSH is really a test hook, so it doesn't need to support
completely random values.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d67d8ed4 by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Shut down transaction_loop clients more cleanly

A transaction_loop client can exit with a transaction active when its
time limit expires.  This causes a recovery and causes problems with
the test cleanup, which detects unwanted recoveries and fails.

Set a flag when the time limit expires and exit cleanly before the
next transaction is started.

Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
52dcecbc by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Add extra debug to large database recovery test

This test sometimes fails, probably because the test is flakey.
Either the records aren't being added correctly or the counting of
records loses records.  Try to debug both possibilities.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
80f3f7c1 by Martin Schwenke at 2018-10-08T03:36:11Z
ctdb-tests: Improve counting of database records

Record counts are sometimes incomplete for large databases when
relevant tests are run on a real cluster.

This probably has something to do with ssh, pipes and buffering, so
move the filtering and counting to the remote end.  This means that
only the count comes across the pipe, instead of all the record data.

Instead of explicitly excluding the key for persistent database
sequence numbers, just exclude any key starting with '_'.  Such keys
are not used in tests.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Oct  8 05:36:11 CEST 2018 on sn-devel-144

- - - - -
f5c6bd5c by David Mulder at 2018-10-08T16:17:14Z
gpo: abstract methods are defined in the parent class

These methods don't need redefined in the child
class.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
5dddb784 by David Mulder at 2018-10-08T16:17:14Z
gpo: Initialize SamDB in the gp_sec_ext

The SamDB is only used by the gp_sec_ext, and
isn't needed elsewhere, so initialize it where
we need it and avoid passing it around
everywhere.
It makes the most sense to put this in the setter
class that uses it, so pass our creds down so we
have access to it, then initialize it there.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
aa9b07ba by David Mulder at 2018-10-08T16:17:14Z
gpo: gp_sec_ext should check whether to apply

Whether an extension should apply should be
determined by the extension, not by the
calling script.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
f702ad94 by David Mulder at 2018-10-08T16:17:15Z
gpupdate: Remove the unnecessary url parameter

The samdb object isn't initialized here anymore,
but in the gp_sec_ext, so this parameter to
gpupdate does nothing.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
4354071b by David Mulder at 2018-10-08T16:17:15Z
gpo: Initialize gp_ext variables in constructor

Initialize variables for the gp_ext in the
constructor instead of passing them via the parse
function.
This is a dependency of the "gpo: Implement
process_group_policy() gp_ext func" patch, since
the parse() function is now called by the ext,
instead of by gpupdate within apply_gp(). The
parse() function should only take the path
variable, to simplify writing Client Side
Extensions.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
fb22582a by David Mulder at 2018-10-08T16:17:15Z
gpo: Remove unused methods from gp_sec_ext

These functions were added by Luke, but have
never actually done anything. If/when we
read from these *.pol files, we won't need these
separate functions to do it.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
7bb326a6 by David Mulder at 2018-10-08T16:17:15Z
gpo: Implement process_group_policy() gp_ext func

MS spec describes the policy callback as a
function called ProcessGroupPolicy which accepts
a pDeletedGPOList and a pChangedGPOList param.
The Group Policy Client Side Extension then
iterates over the deleted, then the changed gpo
lists and applies/unapplies policy. We should do
this also.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
7cef6953 by David Mulder at 2018-10-08T16:17:15Z
gpo: Remove unused gp_ext.list() function

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
78601b35 by David Mulder at 2018-10-08T16:17:15Z
gpo: Move policy application to the gp_ext

Policy specific setting application should be
handled by the group policy extension, not the
read/parse handler.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
96ffc969 by David Mulder at 2018-10-08T16:17:15Z
gpo: remove unreached non-DC branch in gp_sec_ext.apply_map()

We don't get this far if we are not a DC, and if somehow we do the
errors will be no more informative due to this special case.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
8a21ae62 by David Mulder at 2018-10-08T16:17:15Z
gpo: apply_map should not be required for gp_ext

The apply_map function should not be a requirement
to implement the gp_ext class, since only the
gp_sec_ext uses it now.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
cc02de6b by David Mulder at 2018-10-08T16:17:15Z
gpo: Create a function for returning applied settings

This returns a list of guids for gpos applied
plus settings applied and their previous values.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
9d7a0bb3 by Douglas Bagnall at 2018-10-08T16:17:15Z
gpo: avoid quadratic behaviour in guid retrieval

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: David Mulder <dmulder at suse.com>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
30f6ac9c by David Mulder at 2018-10-08T16:17:15Z
gpo: Use the new process_group_policy() for unapply

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
cab16505 by David Mulder at 2018-10-08T16:17:15Z
gpo: Calculate deleted gpos and unapply them

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
7d07d8bf by David Mulder at 2018-10-08T16:17:16Z
gpo: add unapply to the gp_sec_ext

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
471089ee by David Mulder at 2018-10-08T16:17:16Z
gpo: Remove unused apply_log_pop() and list() funcs

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
4e98d18b by David Mulder at 2018-10-08T16:17:16Z
gpupdate: Add the --force option

This option forces the reapplication of policy,
and works the same as MS 'gpupdate /force'

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
6ac1445a by David Mulder at 2018-10-08T16:17:16Z
gpupdate: test the new --force option

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
ba87e23b by Douglas Bagnall at 2018-10-08T16:17:16Z
gpo PEP8: balance whitespace around equals

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: David Mulder <dmulder at suse.com>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
cb3eb79e by David Mulder at 2018-10-08T16:17:16Z
gpo: Test the new get_applied functions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
f390dbfa by David Mulder at 2018-10-08T16:17:16Z
gpo: test the get_deleted_gpos_list() function

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
a5a4f979 by David Mulder at 2018-10-08T19:25:59Z
gpo: Test process_group_policy in gp_sec_ext

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

Autobuild-User(master): Aurélien Aptel <aaptel at samba.org>
Autobuild-Date(master): Mon Oct  8 21:25:59 CEST 2018 on sn-devel-144

- - - - -
8f211efb by Volker Lendecke at 2018-10-08T20:17:09Z
tdb_unpack: Convert to size_t for internal calculations

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0c0c79b3 by Volker Lendecke at 2018-10-08T20:17:09Z
tdb_unpack: Correct "len" arg for "B" format

All but one of the users of the "B" format specifier passed in a pointer
to uint32_t instead of what tdb_unpack expected, an "int". Because this
is a purely internal API, change the tdb_unpack function and adjust that
one caller.

To reviewers: Please check carefully, thanks :-)

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ba787dc4 by Volker Lendecke at 2018-10-08T20:17:09Z
tdb_unpack: Protect against overflow

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1fb2d10d by Volker Lendecke at 2018-10-08T20:17:09Z
registry: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2a35cbb0 by Volker Lendecke at 2018-10-08T20:17:09Z
registry: Add error checks to regdb_fetch_keys_internal

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a9ed0e93 by Volker Lendecke at 2018-10-08T20:17:09Z
registry: Add error checks to regdb_unpack_values

This makes "regdb_unpack_values" take a size_t as buflen. The only
caller calls it with TDB_DATA.dsize, which *is* size_t. Convert the
internal "len" variable to the unsigned size_t as well and add overflow
checks. This depends on tdb_unpack to either return -1 or a positive
value less than or equal to the passed-in "size_t" buflen;

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f20d018a by Volker Lendecke at 2018-10-08T20:17:10Z
registry: Print failure of regdb_unpack_values

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5ba7b204 by Volker Lendecke at 2018-10-08T20:17:10Z
registry: Don't use an uninitialized value

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b83763d1 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Add tdb_chainwalk_check

This captures the tdb_rescue protection against circular hash chains
with a slow pointer updated only on every other record traverse

If a hash chain has a loop, eventually the next_ptr
will cycle around and be identical to the 'slow' pointer.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e63f7bd3 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Make tdb_find circular-safe

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6502f7a3 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Make tdb_dump_chain circular-list safe

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e02c4a41 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Make tdb_find_dead circular-safe

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ade339c8 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Make get_hash_length circular-safe

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
75e79ca5 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7e1ad4c5 by Volker Lendecke at 2018-10-08T20:17:11Z
tdb: Make the freelist walk circular-safe

We can't really do the full check while the freelist is modified on the
fly. As long as we don't merge any freelist entries, we should be good
to apply this check.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
bdacc41f by Volker Lendecke at 2018-10-08T20:17:11Z
tdb: Basic test for circular hash chain fix

This just walks tdb_find by searching for a nonexistent record

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f36a538c by Volker Lendecke at 2018-10-08T20:17:11Z
tdb: Basic test for circular freelist fix

Try to store a record for which the (circular) freelist does not have
any entry.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e7d3678d by Volker Lendecke at 2018-10-08T20:17:11Z
lib: Avoid the use of open_memstream in tevent_req_profile_string

Solaris does not have it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13629
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2329518f by Volker Lendecke at 2018-10-08T20:17:11Z
pdb: Use "sid_compose" where appropriate

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f6137454 by Volker Lendecke at 2018-10-08T20:17:11Z
pdb: Fix some "(ret == true)" to just "(ret)"

"ret" is a boolean, so this should not change semantics

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d25f88f7 by Volker Lendecke at 2018-10-08T23:22:53Z
pdb: Reduce code duplication in make_user_info()

10 lines less and a few hundred (-O0) bytes .text less

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Oct  9 01:22:53 CEST 2018 on sn-devel-144

- - - - -
d46a5d63 by Volker Lendecke at 2018-10-09T20:32:12Z
build: Multi-line deps

We'll add one in the next commit

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9e26f6de by Volker Lendecke at 2018-10-09T20:32:12Z
auth: Use the zlib version of crc32

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dfa51233 by Volker Lendecke at 2018-10-09T20:32:12Z
drsuapi: Use the zlib version of crc32

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
37f6e2a3 by Volker Lendecke at 2018-10-09T23:41:52Z
lib: Remove lib/crypto/crc32.[ch]

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Oct 10 01:41:52 CEST 2018 on sn-devel-144

- - - - -
ea6421d7 by Joe Guo at 2018-10-10T04:16:21Z
emulate/traffic: allow traffic_replay to run users and groups generate multiple times

When we run `traffic_replay --generate-users-only`, if we cancel it or
it breaks in middle, it won't do anything when we try to run it again.

This is because the code will check the first user/group to create. If
it's already there, then it thought task already done, and break the loop.

This commit change the behavior:
We search existing users/groups first, skip existing ones, and
create non-existing ones. So we can run it multi-times to make sure the
expected users and groups are actually created.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e5cf5a68 by Joe Guo at 2018-10-10T04:16:21Z
samba/logger: add logger module for python

We need a consitent way for logging in Samba Python code.

This module provides a factory method `get_samba_logger` to create logger,
with a reasonable default format and optional color.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e8a1773c by Joe Guo at 2018-10-10T04:16:21Z
emulate/traffic: apply new logger to replace print

These print are actually progress infomation, should use logger to
print to stderr, other than stdout.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
33ce1fa2 by Joe Guo at 2018-10-10T04:16:21Z
script/traffic_replay: print packets data to stderr

This is debug info, should print to stderr.
Otherwise it will flood stdout.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
57594c87 by Joe Guo at 2018-10-10T04:16:21Z
script/traffic_replay: get debug level via api

The -d option will set samba global debug level automatically.
We should not parse and use the passed in value.

Use samba.get_debug_level instead.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4260fb87 by Joe Guo at 2018-10-10T04:16:21Z
script/traffic_replay: apply new logger to replace print

Use logger to replace print

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0979d960 by Joe Guo at 2018-10-10T04:16:22Z
netcmd: modify Command.get_logger to use get_samba_logger

By doing this, we don't need to repeat the log level convert code any more.
Also, logs have colors now.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
126a14b4 by Joe Guo at 2018-10-10T04:16:22Z
netcmd: apply the new get_logger to cmds

This is an example of how to use the new logger.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
473c1c00 by Douglas Bagnall at 2018-10-10T04:16:22Z
popt_common_creds: actually use the ignore_missing_conf flag

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Pair-programmed-with: Olly Betts <olly at survex.com>
Reviewed-by: Noel Power <nopower at suse.com>

- - - - -
77ef686f by Gary Lockyer at 2018-10-10T04:16:22Z
pyauth: Remove imessaging_ctx parameter to new

The pyauth code assumes the messaging context code is a py_talloc
object.  But the code in pymessaging returns a wrapped talloc object.
Removing the parameter as it's not currently used by any code.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Noel Power <nopower at suse.com>

- - - - -
79e1ed1f by Douglas Bagnall at 2018-10-10T04:16:22Z
python/upgradehelpers: use int not long for PY3

int works OK for py2 also.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <nopower at suse.com>

- - - - -
131f98ed by Douglas Bagnall at 2018-10-10T07:28:20Z
tests/python/ldap: use int instead of long for time_t

Python int is at least a C long; Python long disappears in Py3.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <nopower at suse.com>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Wed Oct 10 09:28:20 CEST 2018 on sn-devel-144

- - - - -
8b972848 by Ralph Boehme at 2018-10-10T20:22:12Z
s4:torture: FinderInfo conversion test with AppleDouble without xattr data

This testcase demonstrates that the AppleDouble conversion in vfs_fruit
doesn't correctly convert the FinderInfo data from the AppleDouble file
to a stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7e010abb by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: fix two comments

Thanks to the recent addition of ad_convert_xattr() we now correctly
handle this case.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8ee7e613 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: store filler bytes from AppleDouble file header in struct adouble

This can later be used to distinguish between macOS created AppleDouble
files and AppleDouble files created by Samba or Netatalk.

macOS:    "Mac OS X        "
Samba:    "Netatalk        "
Netatalk: "Netatalk        "

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d161e047 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: move setting ADEID_FINDERI length to ad_convert_xattr()

ad_convert_xattr() does the conversion of the xattr data in the
AppleDouble file, so we should update it's size there and should not
defer it to the caller.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8bc36d72 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: do direct return from error checks in ad_convert()

Subsequent commits will move the mmap() into the subfunctions. This
change just prepares for that.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
99cc9ef8 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: remove unneeded fd argument from ad_convert()

Use the struct adouble member ad_fd instead of passing it as an
argument. Who did that in the first place? :)

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b355a095 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: move storing of modified struct adouble to ad_convert()

ad_convert() modified it, so let ad_convert() also save it to disk. No
change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d27d0326 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: move FinderInfo conversion to helper function and call it from ad_convert()

No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4f1174b6 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: move FinderInfo lenght check to ad_convert()

The final step in consolidating all conversion related work in
ad_convert(). No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
98bd7c0a by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: split out truncating from ad_convert()

This may look a little ill-advised as this increases line count, but
the goal here is modularizing ad_convert() itself and making it as slick
as possible helps achieving that goal.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b948681b by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: use ADEDOFF_RFORK_DOT_UND offset macro in ad_convert_truncate()

We really want the fixed size offset here, not a calculated one. Note
that "ad_getentryoff(ad, ADEID_RFORK)" is equal to ADEDOFF_RFORK_DOT_UND
in this case.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4c7e1de4 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: split out moving of the resource fork

No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
93b7e056 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: use ADEDOFF_RFORK_DOT_UND offset macro in ad_convert_move_reso()

We really want the fixed size offset here, not a calculated one. Note
that "ad_getentryoff(ad, ADEID_FINDERI) + ADEDLEN_FINDERI" is equal to
ADEDOFF_RFORK_DOT_UND.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f91e0c85 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: fix error returns in ad_convert_xattr()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
918c6c59 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: let the ad_convert_*() subfunctions mmap as needed

This may mean that we mmap twice when we convert an AppleDouble file,
but this is the only sane way to cleanly modularize ad_convert().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1692ca5f by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: let the ad_convert_*() subfunction update the on-disk AppleDoube header as needed

Another step in simplifying ad_convert() itself. It means that we may
write to disk twice, but is only ever done once per AppleDouble file.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
70d3ae5a by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: call ad_convert_move_reso() from ad_convert_xattr()

ad_convert_xattr() is the place that triggers the need to move the
resource fork, so it should also call ad_convert_move_reso().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5598e6bc by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: add check for OS X filler in FinderInfo conversion

This ensures that the function only acts on AppleDouble files created by
macOS and not AppleDouble files created by us that are already in the
correct format (only using the Resource Fork).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
acb72c1e by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: add out arg "converted_xattr" to ad_convert_xattr

Used to let the caller know if a conversion has been done. Currently not
used in the caller, that comes next.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9cf087a4 by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: make call to ad_convert_truncate() optional

Call ad_convert_truncate() based on whether the previous call
ad_convert_xattr() returned converted_xattr=true.

Upcoming fixes for a different Samba bug (#13642) will hook into calling
ad_convert_truncate() in other cases, this also prepares for that.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
31daab88 by Ralph Boehme at 2018-10-10T23:30:13Z
vfs_fruit: move check in ad_convert() to ad_convert_*() subfunctions

Currently the whole conversion is skipped if the FinderInfo entry in the
AppleDouble file is of the default size (ie not containing xattrs).

That also means we never converted FinderInfo from the AppleDouble file
to stream format. This change finally fixes this.

Note that this keeps failing with streams_depot, much like the existing
known-fail of "samba3.vfs.fruit streams_depot.OS X AppleDouble file
conversion". Fixing the conversion to work with vfs_streams_depot is a
task for another day.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Thu Oct 11 01:30:13 CEST 2018 on sn-devel-144

- - - - -
96b5bf13 by Björn Baumbach at 2018-10-11T08:28:17Z
auth: move copy_session_info() from source3 into the global auth context

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
ea38be48 by Björn Baumbach at 2018-10-11T08:28:17Z
python: Add samba.auth.copy_session_info()

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
9a44be63 by Björn Baumbach at 2018-10-11T08:28:17Z
s4-auth: fix a typo in a comment

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
29af2df4 by Björn Baumbach at 2018-10-11T08:28:17Z
s4-auth: use TALLOC_FREE() shortcut

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
29e757ac by Björn Baumbach at 2018-10-11T08:28:18Z
s4-auth: fetch possible out of memory error

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
f3b7ba17 by Björn Baumbach at 2018-10-11T08:28:18Z
s4-auth: allow to create unix token from system session info

Without this patch security_token_to_unix_token() fails with
NT_STATUS_ACCESS_DENIED, because the system session does only
have one SID.
For a typical token are at least two or more SIDs expected.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
6f08cb66 by Björn Baumbach at 2018-10-11T08:28:18Z
s4-auth: allow to pass original_user_name=NULL to auth_session_info_fill_unix()

With this patch the auth_session_info_fill_unix() uses the "unix_name"
from the session_info->unix_info if no original_user_name was specified.

This is used to process a system session info where no original_user_name
is given.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
ab558fa1 by Björn Baumbach at 2018-10-11T08:28:18Z
pysmbd: add option to pass a session info to set_nt_acl() function

A filled session info is needed by some vfs modules, e.g. full_audit.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
bc8d0d51 by Björn Baumbach at 2018-10-11T08:28:18Z
pysmbd: handle file not found error

Avoid PANIC: internal error

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
38fe315b by Björn Baumbach at 2018-10-11T08:28:18Z
samba-tool ntacl: pass system session to get/set-ntacl functions

The filled session is needed in different vfs modules.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
01ff09ad by Björn Baumbach at 2018-10-11T08:28:18Z
s3/py_passdb: add get_domain_sid() to get domain sid from secrets database

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
e54d4ffb by Björn Baumbach at 2018-10-11T08:28:18Z
samba-tool ntacl: allow to run get/set-ntacl command in non-AD-DC role

Can be used to get and apply NT-ACLs on Samba member servers.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
e5786276 by Björn Baumbach at 2018-10-11T08:28:19Z
selftest: test samba-tool ntacl get/set on AD member server

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
b773be33 by Björn Baumbach at 2018-10-11T08:28:19Z
dns update: add missing newline in error debug message

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
ff3e2fa8 by Björn Baumbach at 2018-10-11T11:40:27Z
vfs_full_audit: ntimes: log a-, m-, c- and creation-time

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Björn Baumbach <bb at sernet.de>
Autobuild-Date(master): Thu Oct 11 13:40:27 CEST 2018 on sn-devel-144

- - - - -
cb5ad7fe by Douglas Bagnall at 2018-10-12T02:16:21Z
s4/script/samba_upgradeprovision: use int not long for Python 3

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1851c35e by Douglas Bagnall at 2018-10-12T02:16:21Z
s4/script/samba_upgradeprovision: remove duplicate (contradictory) dict key

The second, winning, entry says '"defaultSecurityDescriptor": replace + add'

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
69fad8bf by Douglas Bagnall at 2018-10-12T02:16:22Z
s4/script/samba_upgradeprovision: remove unused variable

A similarly named variable is always set two lines down, so we don't need this

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1139a4a6 by Douglas Bagnall at 2018-10-12T02:16:22Z
s4/script/samba_upgradeprovision: set global dnNotToRecalculateFound var

as probably intended. Without this the local variable shadows the
global one and is never used while the global one is never changed.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
dec3eda1 by Tim Beale at 2018-10-12T02:16:22Z
tests: Add corner-case test: fromServer points to dead server

The fromServer attribute is slightly unique, in that it's a DN (similar
to a one-way link), but it is also a mandatory attribute.

Currently, if fromServer gets a bad value (i.e. a dead server that has
been expunged), the DSDB rejects any attempts to modify the associated
nTDSConnection object (regardless of whether or not you're actually
changing the fromServer attribute).

This patch adds a test-case that demonstrates how the DB can get into
such a state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4092b369 by Andrew Bartlett at 2018-10-12T02:16:22Z
dsdb: Ensure that a DN (now) pointing at a deleted object counts for objectclass-based MUST

Add the 'reveal_internals' controls when performing objectclass-based
checks of mandatory attributes. This prevents the extended_dn DSDB
module from suppressing attributes that point to deleted (i.e.
non-existent/expunged) objects.

This ensures that, when modifying an object (and often not even
touching the mandatory attribute) that the fact that an attribute is a
DN, and the DN target is deleted, that the schema check will still pass.

Otherwise a fromServer pointing at a dead server can cause failures,
i.e. you can't modify the affected object at all, because the DSDB
thinks a mandatory attribute is missing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
24669e57 by Tim Beale at 2018-10-12T02:16:22Z
dsdb: Remove redundant variable/check

Previously, this code used to live inside the loop, so the
checked_reveal_control was needed to save ourselves unnecessary work.

However, now that the code has been moved outside the loop, the
checked_reveal_control variable is just unnecessary complication.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
b2c4b829 by Tim Beale at 2018-10-12T02:16:22Z
netcmd: Change Py3 incompatible long() for tombstone expunge

The code to expunge tombstones uses long(), which is not Python3
compatible. Python3 uses int() instead, and works out how big it needs
to be.

As long as we don't run the samba-tool command on a 32-bit machine
after the year 2038, then we should avoid any integer overflow on
Python 2.x.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3efb4588 by Tim Beale at 2018-10-12T05:23:26Z
dsdb: Add dsdb_request_has_control() helper function

Most of the DSDB modules only want to check the existence of a control,
rather than access the control itself. Adding a helper function allows
the code to ask more natural-sounding yes/no questions, and tidies up
an ugly-looking long-line in extended_dn_out.c.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Oct 12 07:23:26 CEST 2018 on sn-devel-144

- - - - -
2fc855e7 by Douglas Bagnall at 2018-10-12T13:27:07Z
samba-tool drs showrepl: do not crash if no dnsHostName found

This should not happen, but it does sometimes in an autobuild
environment. Rather than reporting this by crashing, we report it by
showing there is no DNS name.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Oct 12 15:27:07 CEST 2018 on sn-devel-144

- - - - -
84615c19 by Andreas Schneider at 2018-10-16T06:42:18Z
replace: Add memset_s() if not available

See https://en.cppreference.com/w/c/string/byte/memset

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9291a333 by Andreas Schneider at 2018-10-16T09:38:40Z
s3:lib:popt: Use memset_s() to burn password string

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Tue Oct 16 11:38:40 CEST 2018 on sn-devel-144

- - - - -
fb573c68 by Volker Lendecke at 2018-10-16T16:00:10Z
winbindd_cache: Fix timeout calculation for sid<->name cache

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fdb50817 by Volker Lendecke at 2018-10-16T16:00:10Z
namemap_cache: Absorb the expired calculation into namemap_cache.c

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2557ae53 by Volker Lendecke at 2018-10-16T19:20:19Z
lib: Move the "expired" for gencache_parse calculation into gencache.c

Make it more robust

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Oct 16 21:20:19 CEST 2018 on sn-devel-144

- - - - -
3713905f by Aaron Haslett at 2018-10-17T03:40:06Z
dns: dlz_bind9 reference count logging

dlz_bind9 has to count the number of times the plugin is 'created' by bind's
plugin manager so it doesn't repeat setup.  Logging doesn't reflect this
reference counting logic properly and so messages like "samba_dlz: shutdown"
can, confusingly, come up when the database connection has not actually been
severed.  This patch adds the necessary logging.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13655
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
539daefa by Tim Beale at 2018-10-17T03:40:07Z
libnet/drs: Update replication debug to report link progress

Update the replication debug (for joins/backups) so that it's easier to
see how far through syncing the links we are. E.g. with 150,000 links,
you just get screeds of debug like this, with no real idea how far
through the replication is.

Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[1500/150024]
Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[1500/150024]
Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[1500/150024]

This patch now applies to links the same debug logic we use for objects,
and changes it to look like:

Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[57024/150024]
Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[58524/150024]
Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[60024/150024]

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fdb6b86b by Tim Beale at 2018-10-17T06:56:42Z
drs_util: Improve memory usage when joining large DB

drs_Replicate.replicate() could consume a large amount of memory when
replicating a large DB. This is not a leak - the memory gets freed when
the function returns (i.e. once the partition is fully replicated).
However, while the partition is in the process of being replicated, it
accumulates memory for each replication chunk it receives. This can have
considerable overhead with 1000s of objects/links in the partition.

This was exhausting memory when joining a VM with 1Gb RAM to a DC with
25K users (average ~15 group memberships per user).

It seems that by storing a reference to something that's on the ctr's
talloc tree, it doesn't free up the memory for each ctr message (until
the function actually returns and req is destroyed).

With 10K users (and average 15 group memberships per user), .replicate()
consumed 211Mb of memory, according to talloc.report_full(). With this
patch, it goes down to just the current ctr message (1-2Mb).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Oct 17 08:56:42 CEST 2018 on sn-devel-144

- - - - -
410ec70b by Volker Lendecke at 2018-10-17T17:22:18Z
netsamlogon_cache: Fix talloc_stackframe error return leaks

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9b6166f7 by Volker Lendecke at 2018-10-17T17:22:19Z
netsamlogon_cache: Use "goto fail", save some lines

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
030a3e50 by Volker Lendecke at 2018-10-17T17:22:19Z
netsamlogon_cache: Add some error checks

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fa1f9593 by Volker Lendecke at 2018-10-17T17:22:19Z
netsamlogon_cache: Improve a DBG message

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4df055bb by Volker Lendecke at 2018-10-17T17:22:19Z
auth3: Avoid an explicit ZERO_STRUCT

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2a29ffc3 by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Avoid counting characters manually

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
140a0e05 by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Swap tests: Do cheapest first

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6007c444 by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Call string_term_tdb_data() only once

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
34fe8b1a by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Make gencache_pull_timeout a bit more robust

The previous version assumed a well-formed "val", we just handed it to
strtol without properly checking that it contains the delimiter. So
strtol could well run off the end of "val" in case of data corruption.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
78b8b916 by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Make gencache_pull_timeout return a payload DATA_BLOB

Both relevant callers created one anyway.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0817d107 by Volker Lendecke at 2018-10-17T17:22:20Z
gencache: Remove a redundant check

gencache_pull_timeout checks for NULL ptr already

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
85ec8644 by Volker Lendecke at 2018-10-17T20:22:51Z
gencache: Remove a redundant check

tdb_storev itself is robust against overflow due to multiple buffers

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Oct 17 22:22:51 CEST 2018 on sn-devel-144

- - - - -
d8ea16a3 by Tim Beale at 2018-10-18T04:15:24Z
join: LDAP connection to remote DC can timeout in large join

When joining a very large domain (e.g. 100K users), the replication can
take so long that the LDAP connection to the remote DC times out.

This patch avoids the problem by adding in a sanity-check after the
replication finishes that the LDB connection is still alive. If not,
then we reconnect.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13612

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
30277feb by Andrew Bartlett at 2018-10-18T04:15:24Z
join: Avoid searching for more than strictly required during sanity check

We check for the default base DN as this does require authentication, but
we do not need to search for more than just that (so use SCOPE_BASE) and
we need no attributes, so ask for none

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
10a9cc44 by Tim Beale at 2018-10-18T04:15:24Z
join: Sanity-check LDB connection before failed join cleanup

Joining a large DB can take so long that the LDAP connection times out.
The previous patch fixed the 'happy case' where the join succeeds.
However, if the commit or replication fails (throwing an exception),
then the cleanup code can also fail when it tries to delete objects from
the remote DC. This then gives you an error pointing to
cleanup_old_accounts() rather than what actually went wrong.

This patch adds a sanity-check that if the join fails, that the LDB
connection to the remote DC is still alive, before we start deleting
objects.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13612

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
da875eda by Joe Guo at 2018-10-18T04:15:24Z
uptodateness: add new module and migrate functions from visualize

Both visualize and drs cmd will have uptodateness functions.
Create a new module to reuse code.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
6ed82c7c by Joe Guo at 2018-10-18T04:15:24Z
uptodateness: migrate more methods from visualize

Move methods from cmd_uptodateness to new module.
Will reuse in drs cmd later.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
ff311f1d by Joe Guo at 2018-10-18T04:15:24Z
netcmd/visualize: rm unused code line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
ac1fba31 by Joe Guo at 2018-10-18T04:15:24Z
uptodateness: extract function get_utdv_edges

Extract function to reuse later.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
bbef7d67 by Joe Guo at 2018-10-18T04:15:25Z
uptodateness: extract function get_utdv_distances

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
90975eb0 by Joe Guo at 2018-10-18T04:15:25Z
uptodateness: extract get_utdv_max_distance

To avoid returning 2 values from get_utdv_distances.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
74f7080b by Joe Guo at 2018-10-18T04:15:25Z
uptodateness: migrate get_kcc_and_dsas as a function

We need to reuse it in drs cmd.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
f4f5f174 by Joe Guo at 2018-10-18T04:15:25Z
uptodateness: add get_utdv_summary function

Get utdv summary from distances matrix and support attr filters.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
219b52cf by Joe Guo at 2018-10-18T04:15:25Z
netcmd/drs: add cmd_drs_uptodateness with json support

Add cmd to print uptodateness summary with json support.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
e404b6aa by Joe Guo at 2018-10-18T08:02:19Z
selftest: add tests for samba-tool drs uptodateness

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Oct 18 10:02:19 CEST 2018 on sn-devel-144

- - - - -
204bd0e4 by Douglas Bagnall at 2018-10-18T08:04:02Z
py3/tests/kcc: turn error into failure for flapping.d/kcc

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
30e9c00f by Douglas Bagnall at 2018-10-18T08:04:02Z
py3_tests/kcc : test_verify can hit KCCError as well as GraphError

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ce518f38 by Douglas Bagnall at 2018-10-18T11:17:30Z
ldb_ldif: be less horribly efficient in debugging

perf said all the time was in strlen.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Thu Oct 18 13:17:30 CEST 2018 on sn-devel-144

- - - - -
a0bad136 by Douglas Bagnall at 2018-10-19T01:43:58Z
ldb_ldif: avoid strlen(NULL)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Fri Oct 19 03:43:58 CEST 2018 on sn-devel-144

- - - - -
b6e45fb4 by Gary Lockyer at 2018-10-19T04:17:25Z
python tests Blackbox: add random_password

Add the random_password method to the BlackboxTestCase class and remove
duplicated copies from other test cases. Also use SystemRandom so that
the generated passwords are more cryptographically sound.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e1eee614 by Gary Lockyer at 2018-10-19T04:17:25Z
dsdb encrypted_secrets tests: Allow "ldb://" in file path

When creating a new user and specifying the local file path of the
sam.ldb DB, it's possible to create an account that you can't actually
login with.

This commit contains tests to verify the bug.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13653

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7b59cd74 by Gary Lockyer at 2018-10-19T07:34:46Z
dsdb encrypted_secrets: Allow "ldb:// and "mdb://" in file path

Correctly handle "ldb://" and "mdb://" schemes in the file path when
determining the path for the encrypted secrets key file.

When creating a new user and specifying the local file path of the
sam.ldb DB, it was possible to create an account that you could not
login with. The path for the key file was incorrectly calculated
for the "ldb://" and "mdb://" schemes, the scheme was not stripped from
the path and the subsequent open of the key file failed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13653

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Oct 19 09:34:46 CEST 2018 on sn-devel-144

- - - - -
e310ad7e by Philipp Gesang at 2018-10-19T11:59:04Z
s3:secrets: clean up sid before storing

SIDs may contain non-zero memory beyond SubAuthorityCount:

    {
    key(15) = "SECRETS/SID/FOO"
    data(68) = "\01\04\00\00\00\00\00\05\15\00\00\00}u@\8C\08\A3\06nx\95\16\FE\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00`F\92\B7\03\00\00\00\18e\92\B7\03\00\00\00 at H\92\B7\00\00\00\00"
    }

These parts are lost when converting to ``string format syntax``
so a roundtrip conversion does not result in the same binary
representation.

Ensure that these never reach the tdb by using an initialized
copy. This allows bitwise comparisons of secrets.tdb after
dumping SIDs as text and reading them back.

Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Oct 19 13:59:04 CEST 2018 on sn-devel-144

- - - - -
5b2c3f2f by Volker Lendecke at 2018-10-19T16:52:50Z
lib: Remove gencache.h from proto.h

It's a pain to recompile the world if gencache.h changes

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Oct 19 18:52:50 CEST 2018 on sn-devel-144

- - - - -
9e03fb60 by David Disseldorp at 2018-10-19T21:11:26Z
talloc: deprecate talloc_set_memlimit()

The memlimit functionality was never utilized by Samba. It adds unneeded
complexity, so flag it as deprecated.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2679dd0f by Andreas Schneider at 2018-10-19T21:11:26Z
s3:registry: Avoid a double-free in reg_perfcount

Found by covscan.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f040d660 by Andreas Schneider at 2018-10-19T21:11:26Z
ndr: Init variables of GUID_from_data_blob()

Found by covscan.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1ca21b98 by Andreas Schneider at 2018-10-19T21:11:26Z
s4:torture: Fix the scope of the req variable in drsuapi test

Found by covscan.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d15a00ba by Andreas Schneider at 2018-10-20T00:17:56Z
s3:smbcontrol: Simplify the return code check

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Oct 20 02:17:56 CEST 2018 on sn-devel-144

- - - - -
a3d12252 by Martin Schwenke at 2018-10-22T04:04:20Z
ctdb-daemon: Return early when refusing to run an event script

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c9e1603a by Martin Schwenke at 2018-10-22T04:04:20Z
ctdb-daemon: Exit if eventd goes away

ctdbd enters a broken state if eventd goes away.  A clean shutdown is
not possible because that involves running events.  Restarting eventd
is possible but this might mask a serious problem and it is possible
that eventd might keep on disappearing.  Just exit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
80549927 by Amitay Isaacs at 2018-10-22T04:04:20Z
ctdb-common: Set close-on-exec for startup fd

The startup_fd should not be propagated to the child processes created
from a daemon.  It should only be used in the daemon code to return the
status of the startup.  Another use of startup_fd is to notify the
parent if the daemon process has exited.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
a1909603 by Amitay Isaacs at 2018-10-22T04:04:20Z
ctdb-event: Check the return status of sock_daemon_set_startup_fd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
fbea9d36 by Martin Schwenke at 2018-10-22T07:27:15Z
ctdb-daemon: Fix valgrind hit in event code

==25741== Syscall param write(buf) points to uninitialised byte(s)
==25741==    at 0x4939291: write (write.c:27)
==25741==    by 0x4868285: sys_write (sys_rw.c:68)
==25741==    by 0x13915D: sock_queue_trigger (sock_io.c:316)
==25741==    by 0x4DE6478: tevent_common_invoke_immediate_handler (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DE64A2: tevent_common_loop_immediate (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DEBE5A: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DEA2D6: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DE57E3: _tevent_loop_once (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x15D1BA: ctdb_event_script_args (eventscript.c:821)
==25741==    by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315)
==25741==    by 0x110642: main (ctdbd.c:393)
==25741==  Address 0x57888a4 is 100 bytes inside a block of size 144 alloc'd
==25741==    at 0x48357BF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25741==    by 0x4B9B7C0: talloc_named_const (in /usr/lib/x86_64-linux-gnu/libtalloc.so.2.1.14)
==25741==    by 0x15CCC6: eventd_client_write (eventscript.c:430)
==25741==    by 0x15CCC6: eventd_client_run (eventscript.c:556)
==25741==    by 0x15CCC6: ctdb_event_script_run (eventscript.c:649)
==25741==    by 0x15D198: ctdb_event_script_args (eventscript.c:812)
==25741==    by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315)
==25741==    by 0x110642: main (ctdbd.c:393)
==25741==

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Oct 22 09:27:15 CEST 2018 on sn-devel-144

- - - - -
d114ca1e by Andrew Bartlett at 2018-10-23T03:50:24Z
selftest: Add expected-value testing for userParameters

This does not means that bugs like https://bugzilla.samba.org/show_bug.cgi?id=11881
are fixed, however we do not wish to cause further issues
without noticing it, eg during python3 fixes for dbcheck.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
f5f8676b by Noel Power at 2018-10-23T03:50:24Z
s4/scripting/bin: PY3 Make sure print statements are enclosed by '()'

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ff3f5411 by Noel Power at 2018-10-23T03:50:24Z
python/samba/tests: PY3 iterable has no sort method

map in python3 returns an iterable, in python2 it returned
a list. Iterable has no sort method, use sort function instead or
construct a list from the iterable so you can use list.sort

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
842f09aa by Noel Power at 2018-10-23T03:50:24Z
samba-tool: PY3 dict view doesn't have sort method,

Can't sort a dict view, create a list from view then use list.sort
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
41200262 by Noel Power at 2018-10-23T03:50:24Z
python/samba/gp_parse: PY3 fdeploy_sids needs to use key method for sort

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dcf787b2 by Noel Power at 2018-10-23T03:50:24Z
s4/scripting: PY3 need to convert cmp funct to key func for sort.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e5382179 by Noel Power at 2018-10-23T03:50:24Z
PY3: In a pure PY3 build filter-subunit was getting called without 'python'

tests were getting called with "| ${src}/selftest/filter-subunit" which
resulted in filter-subunit getting execve'd without a calling python. This
resulted in /usr/bin/python (default python) getting called and subsequent
imports failing.

- - - - -
9f8a570f by Noel Power at 2018-10-23T03:50:25Z
selftest/target: Make sure samba-tool is called with ${PYTHON}

Ensure python scripts are called with the python version that
is defined by $PYTHON

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c8403f33 by Noel Power at 2018-10-23T03:50:25Z
python/samba/tests: make sure samba-tool is called with ${PYTHON}

Ensure python scripts are called with the python version that
is defined by $PYTHON

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c751013f by Noel Power at 2018-10-23T03:50:25Z
testprogs/blackbox: Use PYTHON env variable for calling python scripts

Ensure samba-tool is called with correct python that is
defined by $PYTHON

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
15b2ab04 by Noel Power at 2018-10-23T03:50:25Z
s4/scripting: PY3 Ensure python scripts are run with correct python ver.

As part of port samba4.blackbox.samba3dump to python2/3
make sure test_samba3dump.sh runs samba3dump with $PYTHON which should
define the correct python version.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ee595e9b by Noel Power at 2018-10-23T03:50:25Z
s4/setup/tests: PY3 samba-tool needs to be called with correct python ver.

Ensure samba-tool python version defined by $PYTHON

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ade47b3d by Noel Power at 2018-10-23T03:50:25Z
PY3: Only decode when necessary

- - - - -
9ea6cb18 by Noel Power at 2018-10-23T03:50:25Z
python/samba/netcmd: PY3 only possible to decode bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
95dbe117 by Noel Power at 2018-10-23T03:50:25Z
python/samba/tests: PY2/PY3 port samba.tests.dcerpc.integer

Python3 no longer has a long type so the 'L' postfix is no
longer valid. Additionally in python2 an int that exceeds will
be transparently converted into a long when necessary

- - - - -
23239727 by Noel Power at 2018-10-23T03:50:26Z
python: py_strcasecmp_m & py_strstr_m don't handle unicode properly

py_strcasecmp_m & py_strstr_m use PyArg_ParseTuple() with 's' which
in Py2 tries to decode string with the default (e.g. ascii) encoding

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8d7830b6 by Noel Power at 2018-10-23T03:50:26Z
PY3: convert samba.tests.strings to Py2/Py3

Previously the py2 api for strcasecmp_m/strstr_m required strings/unicode
 but couldn't actually handle unicode with anything other than the default
encoding (e.g. ascii). The c-api as been fixed and the encoding steps
(which were unnecessary and causing errors in PY3) have been removed

- - - - -
600fde58 by Noel Power at 2018-10-23T03:50:26Z
python/samba: add alias for ConfigParser for PY2/PY3 compatability

ConfigParser module changed name to configParser in PY3, additionally
the behaviour regarding interpolation has changed. ConfigParser now
has a default interpolation param whose behaviour demands that '%' is
escaped. To maintain behaviour with the python2 version this default
param needs to be changed. Add some alias(s) and 'shim' Configparser
symbol in samba.compat to cater for this.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
04c118cf by Noel Power at 2018-10-23T03:50:26Z
python/samba/gp_parse: PY2/PY3 compat porting for gp_init.py

Fixes
1) use compat versions of ConfigParser and StringIO
2) open file needs to be opened in binary mode as write_pretty_xml
   routine uses BytesIO() object.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
72c5b270 by Noel Power at 2018-10-23T03:50:26Z
python/samba: use PY3 version of ConfigParser

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4287d119 by Noel Power at 2018-10-23T03:50:26Z
PY3: port samba.tests.samba3sam

- - - - -
9ae5fd65 by Noel Power at 2018-10-23T03:50:26Z
s4/scripting/bin: PY3 Fix exception tuple assignments.

In Python3 to access the exception arguments you need to now use
Exception.args, in Python2 you could access these direcly with the
'except' declaration.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ca12342b by Noel Power at 2018-10-23T03:50:26Z
PY3: relative import fixes

- - - - -
8eae420f by Noel Power at 2018-10-23T03:50:27Z
PY3: ensure StringIO usage is py2/py3 compatible

- - - - -
17728c2f by Noel Power at 2018-10-23T03:50:27Z
python/samba: ldb attribute string fix for wafsamba.tests

- - - - -
e5cac2b8 by Noel Power at 2018-10-23T03:50:27Z
python/samba: misc use of str for ldb.bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
370dcbb7 by Noel Power at 2018-10-23T03:50:27Z
samba_tool: Enclose iterator with list

Really strange bug caused by map being updated while being iterated.
This caused keys to be skipped and inconsistent and incorrect
results from ldapcmp.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
557ff7b7 by Noel Power at 2018-10-23T03:50:27Z
python/samba: samba4.blackbox.dbcheck.release-4-1-0rc3

* Various string related fixed py2/py3
* Fix strange double decode followed by encode code (see comments
  in change)
* Added dump_attr_values, simply printing attribute values (from
  sequence) doesn't work (when using string '%s' format codes in
  existing string). We need to print out string from bytes in PY3
  and fallback to repr(which will print b'blah') if we get a decode error

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0febd003 by Noel Power at 2018-10-23T03:50:27Z
s4/scripting: PY2/PY3 port for samba4.blackbox.upgradeprovision.current

o Fix various ldb attribute that need to be converted to string
o dict has no 'has_key' method
o ndr_unpack needs bytes not string
o b64encode needs bytes (so open file with binary mode)
o StandardError was removed in python3 use Exception instead
o fix octal literals
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f94a8137 by Noel Power at 2018-10-23T03:50:27Z
s4/scripting: Py2/Py3 for samba4.blackbox.upgradeprovision.alpha13

fixup source4/scripting/bin/samba_upgradedns
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1a4e95da by Noel Power at 2018-10-23T03:50:27Z
s4/scripting/bin: PY3 make sure GUID result of format is string

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c377bcd4 by Noel Power at 2018-10-23T03:50:28Z
s4/scripting/bin: PY3 fix samba4.blackbox.upgradeprovision.alpha13

- - - - -
60e7aa86 by Noel Power at 2018-10-23T03:50:28Z
s4/scripting/bin: blobs needs to be bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b142e28e by Noel Power at 2018-10-23T03:50:28Z
python/samba: PY3 fix samba4.blackbox.upgradeprovision.release-4-0-0

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
55f51476 by Noel Power at 2018-10-23T03:50:28Z
selftest/filter: PY3 Make filter-subunit forgiving of decoding errors

samba4.local.ndr for one is a test that outputs string in an encoding
that stdin.readline() guesses to be utf8 (but it isn't) filter subunit
can afford to be forgiving of some random text that can't be decoded as
utf8 so lets do that.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7562a551 by Noel Power at 2018-10-23T03:50:28Z
python/samba/netcmd: PY3 Fix error in samba4.blackbox.schemaupgrade

Getting Exception: must be str, not ldb.bytes error in scheme_upgrade
phase of test
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
09f6b864 by Noel Power at 2018-10-23T03:50:28Z
python/samba: PY3 port samba4.blackbox.functionalprep

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7aa40311 by Noel Power at 2018-10-23T03:50:28Z
python/samba/netcmd: PY3 fix samba.tests.domain_backup_offline

Fix attributes that need to be treated as str not bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a114ff21 by Noel Power at 2018-10-23T03:50:29Z
python/samba/samba3: PY3 tdb.Tdb.get method expects bytes in PY3

part of PY3 port samba4.blackbox.upgrade.samba3-upgrade*

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2c0f7f07 by Noel Power at 2018-10-23T03:50:29Z
python/samba/netcmd: PY3 make sure get_testparm_var returns 'str'

part of PY3 port samba4.blackbox.upgrade.samba3-upgrade*

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
86e22c06 by Noel Power at 2018-10-23T03:50:29Z
python/samba/provision: PY3 port samba4.blackbox.provision-backend

Fix some attibrutes that need to be treated as str

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cf38a067 by Noel Power at 2018-10-23T03:50:29Z
python/samba: PY3 port samba4.blackbox.provision-backend

convert ldif content to str when necessary

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c11e90a1 by Noel Power at 2018-10-23T03:50:29Z
python/samba/provision: PY3 PY3 port samba4.blackbox.provision-backend

Enclose filter with list as filter object has no 'len' method

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
25b16fa0 by Noel Power at 2018-10-23T03:50:29Z
python/samba/netcmd: PY3 fix CI error for samba.tests.samba_tool.help

Strangely the test was failing on CI only, looks like there is an
issue with order of elements returned from dict.items() with python3.4
(version of python in CI docker instance) and python3.6 (version on my
development machine). Changed code to sort the keys so order of help
printed out should be the same for each invocation.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f8b5dd95 by Noel Power at 2018-10-23T03:50:29Z
script: Add new (temporary) autobuild task for none-env tests.

Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8ba6bb59 by Noel Power at 2018-10-23T03:50:29Z
Add new CI job for new purepy3-none-env autobuild task

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d3571415 by Noel Power at 2018-10-23T07:10:22Z
CI/Autobuild: Remove samba-none-env-py3 test

We now run a purepython3 none-env test, later when the whole
build is running under python3 we will resurrect this job
but as (samba-none-env-py2) for python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Oct 23 09:10:22 CEST 2018 on sn-devel-144

- - - - -
ec209d28 by Volker Lendecke at 2018-10-25T15:58:22Z
tdb: Avoid casts

We have %PRIu32 and %zu these days

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
c37bf2f9 by Volker Lendecke at 2018-10-25T15:58:23Z
tdb: Use explicit initialization

Let the compiler figure out the optimal code

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
a895cc2a by Volker Lendecke at 2018-10-25T15:58:23Z
tdb: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
8df11518 by Volker Lendecke at 2018-10-25T15:58:23Z
tdb: Remove "USE_RIGHT_MERGES" code

This has not been activated by default for ages and can be very
inefficient. With check_merge_with_left_record() we have an
alternative that will merge freelist records while we walk it
anyway. This has reduced fragmentation significantly

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
afc616b6 by Volker Lendecke at 2018-10-25T15:58:24Z
tdbtorture: No transaction with mutexes

Right now we don't do transactions with mutexed tdbs. tdbtorture -m
locks up. I haven't really investigated why that is the case. The lockup
confused me quite a bit until I figured out it works fine as long as it
does not do transactions, which is all we need right now.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
e2d9abd4 by Volker Lendecke at 2018-10-25T15:58:24Z
tdbtorture: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
34b0f41c by Volker Lendecke at 2018-10-25T15:58:24Z
smbd: Replace some GUID_string by GUID_buf_string

It's only debug statements, but I would like to promote the
stack-allocation routines as good practice where they make sense.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
74de5a5d by Volker Lendecke at 2018-10-25T15:58:24Z
lib: Avoid an "includes.h"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
05954fce by Volker Lendecke at 2018-10-25T15:58:24Z
smbd: Slightly optimize delay_rename_for_lease_break

Do the checks with increasing cost, possibly avoid more expensive ones

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
a03804c4 by Volker Lendecke at 2018-10-25T19:44:17Z
s3:smbd: Move a variable declaration closer to its use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Thu Oct 25 21:44:17 CEST 2018 on sn-devel-144

- - - - -
b3e913ed by Douglas Bagnall at 2018-10-25T19:45:52Z
python/tests/unix: fix spelling and import of text_type

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
3f56dc1a by Douglas Bagnall at 2018-10-25T19:45:52Z
python/tests/gpo: import what we need (errno, not gp_log)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
2ce98901 by Douglas Bagnall at 2018-10-25T19:45:52Z
python/tests/__init__: import what we need

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
24a3acb2 by Douglas Bagnall at 2018-10-25T19:45:52Z
samba-tool drs: remove duplicate and unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
ca40183d by Douglas Bagnall at 2018-10-25T19:45:52Z
samba-tool ldapcmp: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
c3b53264 by Douglas Bagnall at 2018-10-25T19:45:52Z
samba-tool user: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
fe6daff2 by Douglas Bagnall at 2018-10-25T19:45:53Z
samba-tool domain: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
db6ab2cf by Douglas Bagnall at 2018-10-25T19:45:53Z
python/domain_update: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
daea7c1b by Douglas Bagnall at 2018-10-25T19:45:53Z
python/forest_update: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
74a5448b by Douglas Bagnall at 2018-10-25T19:45:53Z
python/gp_ext_loader: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
8519502c by Douglas Bagnall at 2018-10-25T19:45:53Z
python/ntacls: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
9d24ded2 by Douglas Bagnall at 2018-10-25T19:45:53Z
python/schema: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
1ecdde4f by Douglas Bagnall at 2018-10-25T19:45:53Z
python/gp_parse/gp_pol: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
f5df55d2 by Douglas Bagnall at 2018-10-25T19:45:53Z
python/provision: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
018c0138 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/dns*: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
0c18c373 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/dsdb_schema_attr: remove unused/duplicate imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
4fc99a04 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/getdcname: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
eac85232 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/netbios: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
b9161e52 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/*: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
135a3c29 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/samdb: avoid useless local variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
68b284f6 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
5773290a by Douglas Bagnall at 2018-10-25T19:45:54Z
script/generate_param.py: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
dd99bfd8 by Douglas Bagnall at 2018-10-25T19:45:55Z
wintest/test-s3: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
6b88685f by Douglas Bagnall at 2018-10-25T19:45:55Z
ldb-samba/tests/match-rules: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
eef11c44 by Douglas Bagnall at 2018-10-25T19:45:55Z
ldb/tests/_ldb_text: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
02ed2599 by Douglas Bagnall at 2018-10-25T19:45:55Z
talloc/test/pytalloc: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
87272cf1 by Douglas Bagnall at 2018-10-25T19:45:55Z
tdb/test/_tdbtext: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
cb2c5843 by Douglas Bagnall at 2018-10-25T19:45:55Z
tests/blackbox/py: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
ff1bff9e by Douglas Bagnall at 2018-10-25T19:45:55Z
py/tests: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
742c786d by Douglas Bagnall at 2018-10-25T19:45:55Z
s4/dsdb/pytest/dirsync: do not double import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
682b223d by Douglas Bagnall at 2018-10-25T19:45:55Z
s4/dsdb/pytest/dsdb_schema_info: do not double import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
86aca4db by Douglas Bagnall at 2018-10-25T19:45:56Z
s4/dsdb/pytest/: unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
3d5ba7fc by Douglas Bagnall at 2018-10-25T19:45:56Z
auth/cred/tests/bind: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
da9fe70d by Douglas Bagnall at 2018-10-25T19:45:56Z
samba-tool: samba.getopt is not used here

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
e9a57a5b by Douglas Bagnall at 2018-10-25T19:45:56Z
s4/scripting/pfm_verify: remove duplicate import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
b90b2fcd by Douglas Bagnall at 2018-10-25T19:45:56Z
python/tests/kcc.graph: avoid import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
bbf1740b by Douglas Bagnall at 2018-10-25T19:45:56Z
python/tests/kcc_*: avoid * imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
aff2afd4 by Douglas Bagnall at 2018-10-25T19:45:56Z
selftest/perftests: avoid import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
c9c3a9ce by Douglas Bagnall at 2018-10-25T19:45:56Z
python/tests/common: avoid import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
8818b21e by Douglas Bagnall at 2018-10-25T19:45:57Z
selftest/tests.py: avoid import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
9f7247d8 by Douglas Bagnall at 2018-10-25T19:45:57Z
s3/selftest/tests: don't use import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
91bab6f8 by Douglas Bagnall at 2018-10-25T19:45:57Z
s4/selftest/tests: don't use import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
a1450f2c by Douglas Bagnall at 2018-10-25T19:45:57Z
pytest/dcerpc.integer: force py2 long int without incompatible syntax

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
2412ca51 by Douglas Bagnall at 2018-10-25T19:45:57Z
python tests: always use Python's unicodedata

We had our own special version with very few entries, but only
used it in one place.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
bc8201bd by Douglas Bagnall at 2018-10-25T19:45:57Z
samba-tool ldapcmp: use ValueError, not obsolete StandardError

The error is in the value, and StandardError is not in Python 3

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
71825bc9 by Douglas Bagnall at 2018-10-25T19:45:57Z
python/samba/common: py3 compat raw_input

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
d6072cbf by Douglas Bagnall at 2018-10-25T19:45:57Z
python/remove_dc: use a local variable in offline_remove_server

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
159fdcb6 by Douglas Bagnall at 2018-10-25T19:45:58Z
python/kcc: use compat.cmp_fn (PY3)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
cd5ac176 by Douglas Bagnall at 2018-10-25T19:45:58Z
script/show_test_time: approach python 3 compatibility

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
41844d13 by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/dsdb/pytests: Py3 compatitble except clauses

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
d121c68a by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/dsdb/pytest/ad_dc_medley: do not use xrange

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
4fdccc0d by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/dsdb/pytest/sort: use compat.cmp_fn instead of cmp

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
6ab7a3be by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/scripting: py3 style 0o123 octal, not 0123

this works with py2.6+ too.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
105e14cb by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/scripting/demodirsync: fix syntax error

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
bab4f3a1 by Douglas Bagnall at 2018-10-25T19:45:58Z
python/gp_parse/gp_inf: remove shadowed method

The 'from_xml()' definition is replaced a few lines down

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
f12fd900 by Douglas Bagnall at 2018-10-25T19:45:58Z
python/gp_parse/gp_inf: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
03f45656 by Douglas Bagnall at 2018-10-25T19:45:59Z
python/tests/lsa_string: remove duplicate method

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
d042a300 by Douglas Bagnall at 2018-10-25T19:45:59Z
python/tests/kcc_utils: disambiguate/unshadow a test

Pair-programmed-with: Garming Sam <garming at catalyst.net.nz>
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
d2fadfd6 by Douglas Bagnall at 2018-10-25T19:45:59Z
ldb/tests/py/api: reveal shadowed casefold test

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
31172794 by Douglas Bagnall at 2018-10-25T19:45:59Z
s4/dsdb/pytest/ldap_schema: remove duplicate test

The tests differ in two lines, thus:

     def test_subClassOf(self):
     -        """ Testing usage of custom child schamaClass
     +        """ Testing usage of custom child classSchema

[...]

   -governsId: 1.3.6.1.4.1.7165.4.6.2.6.3.""" + str(random.randint(1, 100000)) + """
   +governsId: 1.3.6.1.4.1.7165.4.6.2.6.7.""" + str(random.randint(1, 100000)) + """

with the governsId OID changed because it was noticed they were colliding.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
99be8e98 by Douglas Bagnall at 2018-10-25T19:45:59Z
python/tests/raw_protocol: reveal shadowed test via disambiguation

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
add4068f by Douglas Bagnall at 2018-10-25T19:45:59Z
samba-tool tests: fix bytes/str issue in masked test

This test will be revealed to the world in the next commit.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
654cc08c by Douglas Bagnall at 2018-10-25T19:45:59Z
python/tests/ou: unshadow a test

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
adf73d34 by Douglas Bagnall at 2018-10-25T22:50:37Z
pytests/samba3sam: unshadow and fix a search_non_mapped test

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Fri Oct 26 00:50:37 CEST 2018 on sn-devel-144

- - - - -
525b19fa by Christian Ambach at 2018-10-26T04:59:08Z
s3:script/tests reduce code duplication

Signed-off-by: Christian Ambach <ambi at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b89732c3 by Christian Ambach at 2018-10-26T04:59:09Z
s3:utils/smbget add error handling for mkdir() calls

Signed-off-by: Christian Ambach <ambi at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
fce0d1b2 by Christian Ambach at 2018-10-26T07:58:07Z
s3:utils/smbget fix recursive download with empty source directories

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13199
Signed-off-by: Christian Ambach <ambi at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Oct 26 09:58:07 CEST 2018 on sn-devel-144

- - - - -
ec4b2ac5 by Andreas Schneider at 2018-10-27T19:24:23Z
s3:selftest: Fix test names of smbtorture_s3.plain

The env name will be appended. There is no need to have it twice. Can't
we remove the tests againa ad_dc_ntvfs completely?

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4db918b4 by Andreas Schneider at 2018-10-27T19:24:23Z
s3:torture: Don't use the same testdir twice

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d48a2bc3 by Andreas Schneider at 2018-10-27T19:24:23Z
s3:torture: Rename the test file and remove it if it exists

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
94f8a690 by Volker Lendecke at 2018-10-27T19:24:24Z
lib: Remove unused tdb_trans_* functions

The transactions have all moved to dbwrap

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5391e21d by David Mulder at 2018-10-27T22:35:34Z
lib:socket: If returning early, set ifaces

Prevents a segfault in load_interfaces() when total interfaces == 1.
Fixes regression caused by da68a1b2f417ec82ea4ed3e7a4d867cef8ca8f93.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13665

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Ralph Böhme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Sun Oct 28 00:35:35 CEST 2018 on sn-devel-144

- - - - -
ba17cae4 by Andreas Schneider at 2018-10-29T19:09:25Z
s3:winbind: Check return code of initialize_password_db()

See https://retrace.fedoraproject.org/faf/reports/1577174/

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13668

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0b60afec by Douglas Bagnall at 2018-10-29T19:09:25Z
python/samdb: properly use property()

Python's property() function works like this:

property([getter[, setter[, delete[, doc]]]])

but we have been forgetting the delete function, or rather setting it
to be a string. A string is not callable and is unlikely to succeed at
deleting the property.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b37f8f88 by Douglas Bagnall at 2018-10-29T22:13:36Z
python: do not use "is" for string equality

This is not always going to work, and is not guaranteed to be
consistent even between minor versions.

Here is a simple counterexample:

>>> a = 'hello'
>>> a is 'hello'
True
>>> a is 'hello'.lower()
False
>>> a == a.lower()
True

Possibly it always works for the empty string, but we cannot rely
on that.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Mon Oct 29 23:13:36 CET 2018 on sn-devel-144

- - - - -
5d565f63 by Ralph Boehme at 2018-10-29T22:36:24Z
vfs_fruit: remove check for number of xattrs from ad_convert_xattr

Turns out that there exist AppleDouble files with an extended FinderInfo
entry that includes the xattr marshall buffer, but the count of xattrs
in the buffer is just zero.

We do want to discard this extended FinderInfo entry and convert it to a
simple fixed size FinderInfo entry, so remove the check.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
14abead3 by Volker Lendecke at 2018-10-29T22:36:24Z
tdb: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c18e2ed0 by Volker Lendecke at 2018-10-29T22:36:24Z
tdb: Align an integer type

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
96f6768e by Volker Lendecke at 2018-10-29T22:36:24Z
tdb: Don't delete dead records in traverse

The next commit will change the handling of dead records, removing the
"tdb_do_delete" function. As traverses should not happen in normal
operations, dead records from them should be rare, and relying on
traverses to remove them is a very bad idea IMHO.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4ed2a67a by Volker Lendecke at 2018-10-29T22:36:25Z
tdb: Purge dead records whenever we block the freelist

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
05212658 by Volker Lendecke at 2018-10-29T22:36:25Z
tdb: Do early RDONLY error check for tdb_delete

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
500a729a by Volker Lendecke at 2018-10-30T01:48:38Z
tdb: Make record deletion circular-chain safe

Before this patch we had 3 loops walking a hash chain to delete
records:

tdb_do_delete() to find the predecessor of the record that was to be
deleted. tdb_count_dead(), the name says it all and tdb_purge_dead()
to give back all dead records from a chain to the freelist.

This patch introduces tdb_trim_dead that walks a hash chain just
once. While it does so it counts the number of dead records, and all
records beyond tdb->max_dead_records are moved to the freelist.

Normal record deletion now works by always marking a record as dead in
step 1 and then calling tdb_trim_dead. This is made safe against
circular chains by doing the slow chain walk only in the case when we
did not delete a dead record during our walk.

It changes our dynamics a bit:

When deleting a record with non-zero max_dead_records, now we always
leave that number of records around when deleting, doing a blocking
lock on the freelist when we found too many dead records.

Previously when exceeding max_dead_records we wiped all dead records
to start accumulating them from scratch, assuming we could lock the
freelist in a nonblocking fashion.

The net effect for an uncontended freelist is the same: In
tdb_allocate() we still completely hand over all dead records to the
freelist when we could lock it, it just happens later than without
this patch.

This means for a lightly loaded system we will potentially leave more
dead records around in databases like locking.tdb. However, on a
heavily loaded system we become more predictable: If the freelist is
so heavily contended that across many deletes we can't get hold of it,
previously we accumulated more dead records than max_dead_records
would allow. This is a really lowlevel tradeoff that is likely hard to
measure, but to me becoming more deterministic without sacrificing too
much parallelism (we keep more dead records around) is worth trying.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Oct 30 02:48:38 CET 2018 on sn-devel-144

- - - - -
0189f23f by Stefan Metzmacher at 2018-10-30T06:30:20Z
schema_samba4.ldif: add allocation of DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME

This was already allocated in source4/dsdb/samdb/samdb.h with
commit 22208f52e6096fbe9413b8ff339d9446851e0874.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
60131b44 by Stefan Metzmacher at 2018-10-30T06:30:20Z
s4:dsdb: fix comment on DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
364ed537 by Stefan Metzmacher at 2018-10-30T06:30:20Z
testprogs/blackbox: add samba4.blackbox.test_primary_group test

This demonstrates the bug, that happens when the primaryGroupID
of a user is changed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bb9c9e49 by Stefan Metzmacher at 2018-10-30T06:30:20Z
s4:dsdb: add DSDB_CONTROL_DBCHECK_FIX_LINK_DN_SID oid

This will be used to fix missing <SID=> components in future.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c5c99b56 by Stefan Metzmacher at 2018-10-30T06:30:20Z
dbchecker: improve verbose output of do_modify()

This makes it easier to debug dbcheck problems.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a801799e by Stefan Metzmacher at 2018-10-30T06:30:20Z
dbchecker: Fix missing <SID=...> on linked attributes

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f81771c8 by Stefan Metzmacher at 2018-10-30T06:30:20Z
blackbox/dbcheck-links: Test broken links with missing <SID=...> on linked attributes

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
966c7feb by Stefan Metzmacher at 2018-10-30T06:30:20Z
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_handle_linked_attribs()

This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
42e69a86 by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_add()

This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
70a306d0 by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: add missing \n to a DEBUG message in replmd_modify_la_add()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
738b52eb by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_delete()

This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1ef145d9 by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_replace()

This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0386307e by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: add support for DSDB_CONTROL_DBCHECK_FIX_LINK_DN_SID

This will be used by dbcheck in the next commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7a36cb30 by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:samldb: internally use extended dns while changing the primaryGroupID field

This is important, otherwise we'll loose the <SID=> component of the
linked attribute.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
852e1db1 by Andrew Bartlett at 2018-10-30T09:32:51Z
dsdb: Add comments explaining the limitations of our current backlink behaviour

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Tim Beale <timbeale at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Oct 30 10:32:51 CET 2018 on sn-devel-144

- - - - -
eeb4089d by Gary Lockyer at 2018-10-30T15:40:13Z
dsdb group_audit: Test to replicate BUG 13664

The group audit code incorrectly logs member additions and deletions.

Thanks to metze for the debugging that isolated the issue, and for
suggesting the fix to dn_compare.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8420a4d0 by Gary Lockyer at 2018-10-30T15:40:13Z
dsdb group audit: align dn_compare with memcmp

Rename the parameter names and adjust the  return codes from dn_compare
so that:
dn_compare(a, b) =>

LESS_THAN means a is less than b.
GREATER_THAN means a is greater than b.

Thanks to metze for suggesting the correct semantics for dn_compare

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
e2970887 by Gary Lockyer at 2018-10-30T15:40:13Z
dsdb group audit tests: check_timestamp improve diagnostics

Change check_timestamp to display the expected, actual along with the
line and name of the failing test, rather than the line in
check_timestamp.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
c952fc12 by Gary Lockyer at 2018-10-30T15:40:13Z
dsdb group audit tests: check_version improve diagnostics

Change check_version to display the expected, actual along with the
line and name of the failing test, rather than the line in check_version

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
aeef8b41 by Gary Lockyer at 2018-10-30T19:20:26Z
dsdb group audit tests: log_membership_changes extra tests

Add extra tests to ensure better test coverage of log_membership_changes

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Tue Oct 30 20:20:26 CET 2018 on sn-devel-144

- - - - -
b40daca6 by Tim Beale at 2018-10-30T23:30:16Z
traffic_replay: Generate users faster by writing to local DB

We can create user accounts much faster if the LDB connection talks
directly to the local sam.ldb file rather than going via LDAP. This
patch allows the 'host' argument to the tool to be a .ldb file (e.g.
"/usr/local/samba/private/sam.ldb") instead of a server name/IP.

In most cases, the traffic_replay tool wants to run on a remote device
(because the point of it is to send traffic to the DC). However, the
--generate-users-only is one case where the tool can be run locally,
directly on the test DC. (The traffic_replay user generation is handy
for standalone testing, because it also handles assigning group
memberships to the generated user accounts).

Note that you also need to use '--option="ldb:nosync = true"' to get
the improvement in performance.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
49434731 by Tim Beale at 2018-10-30T23:30:16Z
traffic_replay: Change print() to use logger()

This reduces noise, so the messages only come out if you specify
--debug.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
4a8a0ab3 by Tim Beale at 2018-10-30T23:30:16Z
traffic_replay: logger was ignoring smb.conf log-level

We were trying to access the debug-level (in python C bindings) before
the smb.conf had been loaded and actually set the debug-level. So it
would default to zero, regardless of what was in the smb.conf.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5a57378d by Tim Beale at 2018-10-30T23:30:16Z
netcmd: Add backend-store option to domain backup/rename cmds

Currently the online/rename backup files always use the default backend
(TDB) and there is no way to change this.

This patch adds the backend-store option to the backup commands so that
you can create a backup with an MDB backend, if needed.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a9a65adb by Tim Beale at 2018-10-30T23:30:16Z
selftest: Specify different DB backends for restored testenvs

Vary the DB backend that we use for the renamed DCs. The labdc and
renamedc are fairly similar, so let's have each of them use a different
backend.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
ca570bd4 by Tim Beale at 2018-10-30T23:30:16Z
netcmd: Include num-members in 'samba-tool group list --verbose'

This adds an easy way for users to see (via samba-tool) how many members
are in various groups, without querying the members for each individual
group.

For example, you could pipe this output to grep to check for groups with
zero or one members (i.e. historic groups that may no longer make
sense).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
0c910245 by Tim Beale at 2018-10-31T02:40:41Z
netcmd: Add 'samba-tool group stats' command

With large domains it's hard to get an idea of how many groups there
are, and how many users are in each group, on average. However, this
could have a big impact on whether a problem can be reproduced or not.

This patch dumps out some summary information so that you can get a
quick idea of how big the groups are.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Wed Oct 31 03:40:41 CET 2018 on sn-devel-144

- - - - -
bda98b38 by Andreas Schneider at 2018-10-31T20:27:16Z
third_party: Update nss_wrapper to version 1.1.5

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9a4c1888 by Ralph Boehme at 2018-10-31T20:27:16Z
s3:smbd: remove "0x" string prefix from dev/ino

We used %llu as conversion specifier which results in a decimal number
being printed, so remove the misleading "0x" prefix.

While at it, I'll change %llu to the terse %ju.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f7a98094 by Ralph Boehme at 2018-10-31T20:27:17Z
docs:vfs_fruit: add "wipe_intentionally_left_blank_rfork" option

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2a9574b1 by Ralph Boehme at 2018-10-31T20:27:17Z
docs:vfs_fruit: add "delete_empty_adfiles" option

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c3a1f3ec by Ralph Boehme at 2018-10-31T20:27:17Z
s3:selftest: list vfs testssuites one per line

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6f022e61 by Ralph Boehme at 2018-10-31T20:27:17Z
s4:torture: add test for AppleDouble ResourceFork conversion

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2dbb2d28 by Ralph Boehme at 2018-10-31T20:27:17Z
vfs_fruit: add option "wipe_intentionally_left_blank_rfork"

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7be979f9 by Ralph Boehme at 2018-10-31T20:27:17Z
vfs_fruit: detect empty resource forks in ad_convert()

For some reason the macOS client often writes AppleDouble files with a
non-zero sized resource fork, but the resource fork data is just
boilerplate data with the following string close to the start

  This resource fork intentionally left blank

A dump with apple_dump looks like this:

Entry ID   : 00000002 : Resource Fork
Offset     : 00000052 : 82
Length     : 0000011E : 286

-RAW DUMP--:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F : (ASCII)
00000000   : 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 1E : ................
00000010   : 54 68 69 73 20 72 65 73 6F 75 72 63 65 20 66 6F : This resource fo
00000020   : 72 6B 20 69 6E 74 65 6E 74 69 6F 6E 61 6C 6C 79 : rk intentionally
00000030   : 20 6C 65 66 74 20 62 6C 61 6E 6B 20 20 20 00 00 :  left blank   ..
00000040   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000050   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000060   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000070   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000080   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000090   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000A0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000B0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000C0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000D0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000E0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000F0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000100   : 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 1E : ................
00000110   : 00 00 00 00 00 00 00 00 00 1C 00 1E FF FF       : ..............

We can safely discard this Resource Fork data.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e00e6134 by Ralph Boehme at 2018-10-31T20:27:17Z
vfs_fruit: add option "delete_empty_adfiles"

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3649f1a4 by Ralph Boehme at 2018-10-31T20:27:18Z
vfs_fruit: optionally delete AppleDouble files without Resourcefork data

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9f58997d by Ralph Boehme at 2018-10-31T20:27:18Z
vfs_streams_xattr: fix open implementation

Since a long time the modules's open function happily returned success
when opening a non existent stream without O_CREAT.

This change fixes it to return -1 and errno=ENOATTR if

o get_ea_value() returns NT_STATUS_NOT_FOUND (eg mapped from
  getxattr() = -1, errno=ENOATTR) and

o flags doesn't contain O_CREAT

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
242f42ef by Ralph Boehme at 2018-10-31T20:27:18Z
s4:torture/vfs/fruit: skip a few tests when running against a macOS SMB server

These tests are designed to test specific vfs_fruit functionality.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
394d1fc9 by Ralph Boehme at 2018-10-31T20:27:18Z
s4:torture/vfs/fruit: fix a few error checks in "delete AFP_AfpInfo by writing all 0"

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1d4e5064 by Ralph Boehme at 2018-10-31T20:27:18Z
s4:torture/vfs/fruit: set share_access to NTCREATEX_SHARE_ACCESS_MASK in check_stream_list

Avoid sharing conflicts with other opens on the basefile.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dba9a474 by Ralph Boehme at 2018-10-31T20:27:18Z
s4:torture/vfs/fruit: update test "SMB2/CREATE context AAPL" to work against macOS

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
12549886 by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: update test "stream names" to work with macOS

o create the basefile before trying to create a stream on it, otherwise
  this fails on macOS

o write something to the stream, otherwise the stream is not listed as
  macOS hides 0-byte sized streams

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f7551d8f by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: ensure a directory handle is closed in all code paths

Otherwise we get a sharing violation when running against Samba and
opening the directory a second time.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9cd9859d by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: update test "read open rsrc after rename" to work with macOS

macOS SMB server seems to return NT_STATUS_SHARING_VIOLATION in this
case while Windows 2016 returns NT_STATUS_ACCESS_DENIED.

Lets stick with the Windows error code for now in the Samba fileserver,
but let the test pass against macOS.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0d9a80b6 by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: expand existing test "setinfo delete-on-close AFP_AfpInfo" a little bit

Add a check that verifies a create on a stream gets
NT_STATUS_DELETE_PENDING after delete-on-close has been set.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6f428607 by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: expand existing vfs_test "null afpinfo"

This adds a check that a read on a seperate handle also sees the
previously created AFP_AfpInfo stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c3cf09a0 by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: update test "creating rsrc with read-only access" for newer macOS versions

While this operation failed against older macOS versions, it passes
against versions 10.12 and newer. Update the test accordingly, a
subsequent commit will then update our implementation.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
055ca448 by Ralph Boehme at 2018-10-31T20:27:19Z
vfs_fruit: update handling of read-only creation of resource fork

macOS SMB server versions supports this since 10.12, so we adapt our
behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b35d9e74 by Ralph Boehme at 2018-10-31T20:27:20Z
s4:torture/vfs/fruit: expand test "setinfo eof stream"

o Adds checks verifying that after setting eof to 0 on a stream, a
  subsequent open gets ENOENT, before and after closing the handle that
  had been used to set eof to 0.

o Verify that a write to a handle succeeds after that handle has been
  used to set eof to 0 on a stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3195ff95 by Ralph Boehme at 2018-10-31T20:27:20Z
s4:torture/vfs/fruit: write some data to a just created teststream

Doesn't currently make a difference, but this prepares for a later
change in vfs_fruit that will filter out empty streams (which is the
macOS behaviour).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
287082d7 by Ralph Boehme at 2018-10-31T20:27:20Z
vfs_fruit: don't unlink 0-byte size truncated streams

This caused all sort of havoc with subsequent SMB request that acted on
the handle of the then deleted backend storage (file or blob, depending
on the used streams module).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7485007b by Ralph Boehme at 2018-10-31T20:27:20Z
s4:torture/vfs/fruit: enable AAPL extensions in a bunch of tests

These tests check for macOS SMB server specific behaviour. They work
currently against Samba without enabling AAPL because in vfs_fruit we're
currently don't check whether AAPL has been negotiated in one place. A
subsequent commit will change that and this commit prepares for that
change.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
046456fc by Ralph Boehme at 2018-10-31T20:27:20Z
vfs_fruit: use check on global_fruit_config.nego_aapl for macOS specific behaviour

Ensure any non MS compliant protocol behaviour targetted at supporting
macOS clients are only effective if the client negotiated AAPL.

Currently this only guards the resource fork which only macOS client are
going to use, but subsequent commits add more this at this place.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ebfcf75e by Ralph Boehme at 2018-10-31T20:27:20Z
vfs_fruit: filter empty streams

First step in achieving macOS compliant behaviour wrt to empty streams:
- hide empty streams in streaminfo
- prevent opens of empty streams

This means that we may carry 0-byte sized streams in our streams
backend, but this shouldn't really hurt.

The previous attempt of deleting the streams when an SMB setinfo eof to
0 request came in, turned out be a road into desaster.

We could set delete-on-close on the stream, but that means we'd have to
check for it for every write on a stream and checking the
delete-on-close bits requires fetching the locking.tdb record, so this
is expensive and I'd like to avoid that overhead.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
64b28e4a by Ralph Boehme at 2018-10-31T20:27:20Z
s4:torture/util: add torture_smb2_open()

This seems to be missing: a simple wrapper to just open a file without
fancy options.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
aba502d4 by Ralph Boehme at 2018-10-31T20:27:21Z
s4:torture/vfs/fruit: add check_stream_list_handle()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9d8751db by Ralph Boehme at 2018-10-31T20:27:21Z
s4:torture/vfs/fruit: add test "empty_stream"

One to rule them all: consistently test critical operations on all
streams relevant to macOS clients: the FinderInfo stream, the Resource
Fork stream and an arbitrary stream that macOS maps to xattrs when
written to on a macOS SMB server.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
993c7c4e by Ralph Boehme at 2018-10-31T20:27:21Z
vfs_fruit: add some debugging of dev/ino

Aids in debugging dev/ino mismatch failures in open_file_ntcreate.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a26032c3 by Ralph Boehme at 2018-10-31T20:27:21Z
vfs_fruit: remove resource fork special casing

Directly unlinking a file with open handles is not good, don't do it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1e055a79 by Ralph Boehme at 2018-10-31T20:27:21Z
vfs_fruit: add fio->created

fio->created tracks whether a create created a stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
96320ecc by Ralph Boehme at 2018-10-31T20:27:21Z
vfs_fruit: prepare struct fio for fake-fd and on-demand opening

Not used for now, that comes in the subsequent commits.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4a5c9a9e by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: prepare fruit_pwrite_meta() for on-demand opening and writing

This avoid creating files or blobs in our streams backend when a client
creates a stream but hasn't written anything yet. This is the only sane
way to implement the following semantics:

* client 1: create stream "file:foo"

* client 2: open stream "file:foo"

The second operation of client 2 must fail with NT_STATUS_NOT_FOUND.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d7d92710 by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: prepare fruit_pread_meta() for reading on fake-fd

If the read on the stream fails we may have hit a handle on a just
created stream (fio->created=true) with no data written yet.

If that's the case return an empty initialized FinderInfo blob.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
80afafe3 by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: do ino calculation

As we'll start returning fake fds in open shortly, we can't rely on the
next module to calculat correct inode numbers for streams and must take
over that responsibility.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6fd256af by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: let fruit handle all aio on the FinderInfo metadata stream

This will be required to support using fake fds for the FinderInfo
metadata stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6e13dbdd by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: pass stream size to delete_invalid_meta_stream()

delete_invalid_meta_stream() is meant to guard against random data being
present in the FinderInfo stream. If the stream size is 0, it's likely a
freshly created stream where no data has been written to yet, so don't
delete it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
480695cd by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: let fruit_pwrite_meta_stream also ftruncate empty FinderInfo

fruit_streaminfo currently filters out the FinderInfo stream is
delete-on-close is set. We set it here internally, but the client may
also set it over SMB. Turns out that the macOS SMB server does NOT
filter out FinderInfo stream with delete-on-close set, so we must change
the way filtering is done in fruit_streaminfo.

Filtering is now done based on the FinderInfo stream being 0-bytes large which
is why I'm adding the ftruncate here.

No idea why the tests that check the filtering passed the commits
leading up to this one, but if you revert this commit after applying the
whole patchset, the "delete AFP_AfpInfo by writing all 0" test will fail.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d8c1bb52 by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: don't check for delete-on-close on the FinderInfo stream

macOS SMB server doesn't filter out the FinderInfo stream if it has
delete-on-close set.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1b2de44e by Ralph Boehme at 2018-11-01T00:14:23Z
vfs_fruit: let fruit_open_meta() with O_CREAT return a fake-fd

This is the final step in implementing the needed macOS semantics on the
FinderInfo stream: as long as the client hasn't written a non-zero
FinderInfo blob to the stream, there mustn't be a visible filesystem
entry for other openers.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Nov  1 01:14:23 CET 2018 on sn-devel-144

- - - - -
8d14714c by Samuel Cabrero at 2018-11-01T00:59:10Z
s3: winbind: Remove fstring from wb_acct_info struct

The group enumeration backend functions try to allocate an array of
wb_acct_info structs with a number of elements equal to the number of
groups. In domains with a large number of groups this allocation may
fail due to the size of the chunk.

Found while trying to enumerate the groups in a domain with more than
700k groups.

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2cfb58d7 by Ralph Wuerthner at 2018-11-01T00:59:10Z
nsswitch: use goto to have only one function return

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e82b3ac0 by Ralph Wuerthner at 2018-11-01T00:59:10Z
nsswitch: make wb_global_ctx private add add get/put functions to access global context

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
988182c3 by Ralph Wuerthner at 2018-11-01T00:59:10Z
nsswitch: protect access to wb_global_ctx by a mutex

This change will make libwbclient thread safe for all API calls not using a
context. Especially there are no more conflicts with threads using nsswitch
and libwbclient in parallel.

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b5ea7946 by Ralph Wuerthner at 2018-11-01T00:59:10Z
nsswitch: add test for parallel NSS & libwbclient calls

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
40bd0a93 by Volker Lendecke at 2018-11-01T04:06:23Z
nsswitch: Run nsswitch thread test

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Nov  1 05:06:23 CET 2018 on sn-devel-144

- - - - -
371297a5 by Douglas Bagnall at 2018-11-01T04:08:09Z
tests/samba_tool/provision_password_check: follow super inheritance

We were skipping a level in the inheritance chain, which had no effect
in this case (no .setUps or .tearDowns were missed) but it would be
confusing if the parents ever changed.

Note: in python 3, you just call super() with no args, and it works
out the right thing.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
99eee0a6 by Douglas Bagnall at 2018-11-01T04:08:09Z
selftest/format_subunit_json: py3 print()

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
a6368ae6 by Douglas Bagnall at 2018-11-01T04:08:09Z
s4/dsdb/pytest/ldap_schema: fix typo in docs

Commit 311727947799e896e05d644103c9db80a665de88 removed a duplicate
test but it removed the wrong one, leaving this dreadful typo.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
4e694079 by Douglas Bagnall at 2018-11-01T04:08:09Z
tests/python/notification: safer use of super()

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
6c1d6053 by Douglas Bagnall at 2018-11-01T04:08:09Z
tests/python/sec_descriptor: safer use of super()

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
85323088 by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/script/depfilter.py: use py3 compatible regex import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
f109fde3 by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/scripting/autoidl: py3 compatible except

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
2e6fccbb by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/scripting/minschema: whitespace reformat and py3 compatible print

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
b4fff0ac by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/scripting/samba_upgradedns: avoid .has_key()

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
b1ba193d by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/scripting/pfm_verify: PY3: use compat.text_type

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
2013cd28 by Douglas Bagnall at 2018-11-01T04:08:10Z
script/show_test_time: attempt py3 compat

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
2fedb950 by Douglas Bagnall at 2018-11-01T04:08:10Z
python: PY3 Exceptions don't have .message

but str(e) is the same as str(e.message), so we can use that
on 2 and 3.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
9057b1c4 by Douglas Bagnall at 2018-11-01T04:08:10Z
dbcheck: fix message formatting

previously these would have raised an exception

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
e621472c by Douglas Bagnall at 2018-11-01T04:08:10Z
dbcheck: fix function call (right arguments)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
e25878ec by Douglas Bagnall at 2018-11-01T04:08:11Z
samba-tool: let self.usage() find argv[0] by itself

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
ced6b6c9 by Douglas Bagnall at 2018-11-01T04:08:11Z
samba-tool user: fix message format

There were 2 % formats and 3 arguments.
Also reformat for line length

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
cf7f14a4 by Douglas Bagnall at 2018-11-01T04:08:11Z
python/subnets: use the correct variable name

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
5304ea44 by Douglas Bagnall at 2018-11-01T04:08:11Z
python/tests/dsdb: use correct variable names

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
f129ca98 by Douglas Bagnall at 2018-11-01T04:08:11Z
python/tests/dnscmd: don't use undefined name

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
fc64f258 by Douglas Bagnall at 2018-11-01T04:08:11Z
ldb/tests/py/api: use proper name for ldb.LdbError

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
78ead841 by Douglas Bagnall at 2018-11-01T04:08:11Z
ldb/tests/py/index: remove dup value in dictionary

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
9148ec7d by Douglas Bagnall at 2018-11-01T04:08:11Z
s4/dsdb/pytest/sites: do not use variables out of scope

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
4be0bcb1 by Douglas Bagnall at 2018-11-01T04:08:12Z
python join: use the sd_utils we imported

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
7058e21e by Douglas Bagnall at 2018-11-01T04:08:12Z
python: avoid useless work in dsdb_Dn.__cmp__

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
f0c45e08 by Douglas Bagnall at 2018-11-01T04:08:12Z
python dbcheck: use real exception name

and conventional indent

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
d9b45794 by Douglas Bagnall at 2018-11-01T04:08:12Z
Python provision: remove "ExistingBackend"

This was unused and broken. e.g. here:

-    def init(self):
-        # Check to see that this 'existing' LDAP backend in fact exists
-        ldapi_db = Ldb(self.ldapi_uri)

there is no attribute self.ldapi_uri, so this would always raise an
exception.

It was being left around in case it became useful, but that doesn't
seem to be happening.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
f6f98ed2 by Douglas Bagnall at 2018-11-01T04:08:12Z
provision: fix string format syntax

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
67c5ab17 by Douglas Bagnall at 2018-11-01T04:08:12Z
provision: fix string formatting (number of args)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
71f91ccd by Douglas Bagnall at 2018-11-01T04:08:12Z
provision/backend: fix formating syntax

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
7db224bd by Douglas Bagnall at 2018-11-01T04:08:12Z
provision/backend: LDAPBackendResult takes no creds

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
7fafd579 by Douglas Bagnall at 2018-11-01T04:08:13Z
python/remove_dc: fix formating syntax (missing %s)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
7b1d878f by Douglas Bagnall at 2018-11-01T04:08:13Z
python/remove_dc: fix Exception construction

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
d4e06603 by Douglas Bagnall at 2018-11-01T04:08:13Z
waflib: add necessary imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
aafe6bf9 by Douglas Bagnall at 2018-11-01T04:08:13Z
waflib: fix syntax error in string format

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
9c2447e2 by Douglas Bagnall at 2018-11-01T04:08:13Z
py/tests/dcerpc_rpc: Py3 compat integer types

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
9055602e by Douglas Bagnall at 2018-11-01T04:08:13Z
traffic_replay: Exception has no .message

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1a66eeb1 by Douglas Bagnall at 2018-11-01T04:08:13Z
tests/rodc_rwdc: fix message format

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1ca5bf32 by Douglas Bagnall at 2018-11-01T04:08:13Z
python/kcc/graph: import KCCError, which is used somewhere

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
68a43a18 by Douglas Bagnall at 2018-11-01T04:08:14Z
python/samba3: import passdb in the manner it is used

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
866d5b3c by Douglas Bagnall at 2018-11-01T04:08:14Z
samba-tool rodc: remove unused variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
202be599 by Douglas Bagnall at 2018-11-01T04:08:14Z
samba-tool spn: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
d5f1ea60 by Douglas Bagnall at 2018-11-01T04:08:14Z
dbchecker: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d7f26be9 by Douglas Bagnall at 2018-11-01T04:08:14Z
samba-tool dbcheck: remove unused variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
55aaa9c8 by Douglas Bagnall at 2018-11-01T04:08:14Z
python/upgradehelpers: remove unused variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
78f0860c by Douglas Bagnall at 2018-11-01T04:08:14Z
python/xattr: remove unused variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
cf3d35c8 by Douglas Bagnall at 2018-11-01T04:08:14Z
python/tests/credentials: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
e37edb76 by Douglas Bagnall at 2018-11-01T04:08:14Z
python/tests/source: remove useless local variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
5e796308 by Douglas Bagnall at 2018-11-01T04:08:15Z
autobuild: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
4e19b69b by Douglas Bagnall at 2018-11-01T04:08:15Z
py/tests/dcerpc_integer: remove dup tests

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
77b9b5e8 by Douglas Bagnall at 2018-11-01T04:08:15Z
samba-tool domain: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
00eaea56 by Douglas Bagnall at 2018-11-01T04:08:15Z
samba-tool processes: remove unused imports and variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
d1777971 by Douglas Bagnall at 2018-11-01T04:08:15Z
python/upgradehelpers: remove unused provision imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
eeb1e81e by Douglas Bagnall at 2018-11-01T04:08:15Z
python/join: remove unused imports and variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5f19bf63 by Douglas Bagnall at 2018-11-01T04:08:15Z
python/tests/docs: remove unused import, variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
790acef2 by Douglas Bagnall at 2018-11-01T08:40:02Z
samba-tool ntacl: remove unused imports and variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Thu Nov  1 09:40:02 CET 2018 on sn-devel-144

- - - - -
d29d2f2f by Douglas Bagnall at 2018-11-01T08:41:19Z
pytests: allow blackbox subcommands without a shell

When given a list, it will use the list directly as an argument list,
avoiding shell-expansion and the intermediatory process.

This removes shell expansion trouble, and saves the machine a little
bit of work.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1451b696 by Tim Beale at 2018-11-01T08:41:19Z
selftest: Add new customdc testenv that can load any backup-file

This adds a new testenv that can be used for sandpit/manual testing.
This testenv can be based off any backup-file that you like.

The main use case is large databases. Populating 1000s of users is
time-consuming (it can take hours to create a really large DB). Instead
of having to manually add users to the testenv every time you want to
try something, this allows you to populate the users just once, take a
backup/snapshot of the DB, and then spin up the backup multiple times.

In theory this testenv could be useful for other situations too, e.g.
dealing with a corrupted database, testing DB migration (e.g. 4.7 -->
4.8), or if (for some reason) you wanted to create a realistic
lab-domain within a testenv.

To run-up the testenv you need to specify a BACKUP_FILE environment
variable (the same way we specify the SELFTEST_TESTENV), e.g.
  BACKUP_FILE=/files/backup-10k-ad_dc.tar.bz2 \
    SELFTEST_TESTENV=customdc make testenv

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c35fb3a8 by Tim Beale at 2018-11-01T08:41:19Z
dnsupdate: Pass smb.conf through to samba-tool commands

If you call samba_dnsupdate with a --configfile option, this wasn't
passed through to the samba-tool commands the script tries to run.
Normally, samba_dnsupdate would only be run on the DC itself, so it
shouldn't be a big deal, however, this may be a problem if you install
the samba database into a non-default location (i.e. not
/usr/local/samba).

This patch passes through the smb.conf file, if one was specified.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
018ff496 by Tim Beale at 2018-11-01T08:41:20Z
dnsupdate: Skip kerberos step if use-file specified

If there's a problem in get_credentials() (getting the machine account
Kerberos credentials), then we fallback to use_samba_tool (essentially
ignoring use-file). However, there's no need to do this, as use-file
shouldn't require Kerberos credentials.

This was making bootstrapping issues starting a testenv harder to debug.
Obviously, Kerberos is dependent on DNS functioning correctly, but
running dnsupdate was also dependent on having a working Kerberos KDC.
In my case, the testenv had a bad krb5.conf file, but the problem
appeared as resolv-wrapper errors (due to a missing RESOLV_WRAPPER_HOSTS
file, which should've been generated by dnsupdate).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Pair-Programmed-With: Garming Sam <garming at catalyst.net.nz>

- - - - -
e2ee5952 by Tim Beale at 2018-11-01T12:06:01Z
selftest: Remove unnecessary code for backup testenvs

setup_namespaces() already gets done for the backupfromdc's domain, so
this step is unnecessary for the restoredc and offlinebackupdc testenvs
(which are based off the backupfromdc's database).

The setup_namespaces() step is still necessary for the renamedc/labdc,
as these don't have the UPN/SPN suffixes for the new realm yet.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Nov  1 13:06:01 CET 2018 on sn-devel-144

- - - - -
9b28d47b by Volker Lendecke at 2018-11-01T16:34:31Z
torture: Fix the clang build

It's used uninitialized if an early torture_assert fails

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Thu Nov  1 17:34:31 CET 2018 on sn-devel-144

- - - - -
5528a4cc by Tim Beale at 2018-11-01T19:38:14Z
replmd: Add more debug for replicating links

During a join of a large DB, processing the linked attributes can take a
long time. The join hangs in 'Committing SAM database' for many minutes
with no indication of whether it's making progress or not.

This patch adds some extra debug to show how far through processing the
linked attributes we are, when there are many thousands of links.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
628b9f71 by Tim Beale at 2018-11-01T19:38:14Z
replmd: Group together link attribute processing by source object

Instead of processing each link attribute one at a time, we want to
group them together by source object. This will mean we only have to
look-up the source object once, and only perform one DB 'modify'
operation. With groups with 1000s of members, this will help improve
performance.

This patch takes the first step of group together the links by
source-object. A new 'la_group' struct is added to help track what links
belong to the same source object. The la_list essentially becomes a
'list of lists' now.

Note that only related links *in the same chunk* are only grouped together.
While it is trivial to groups together links that span different
replication chunks, this would be a fairly insignificant efficiency gain,
but seems to have a fairly detrimental memory overhead, once you get
into groups with 10,000+ members.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0ba02868 by Tim Beale at 2018-11-01T19:38:14Z
replmd: Split apart source/target checks for links

We've grouped the linked attributes by source-object. Next, we want to
avoid duplicated processing for the source object, i.e. we only need to
check the source object exists once, not once per link.

Before we can do this, we need to tease apart
replmd_extract_la_entry_details(), which is doing both source and target
object processing. Split out extracting the target DSDB-DN so that it's
done separately.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d54956fd by Tim Beale at 2018-11-01T19:38:14Z
replmd: Move talloc context one level up

Eventually we want to combine multiple link attributes, that apply to the
same source object, into a single DB 'modify' operation. This will mean
the memory context needs to hang around until we have performed the DB
operation (instead of allocating a temporary context for each link).

This patch moves the talloc context one level up, so a temp context gets
allocated for each link *group*, instead of for each link *attribute*.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
775054af by Tim Beale at 2018-11-01T22:48:21Z
replmd: Single DB operation per source object during link processing

Move the source object checks and DB modify operation up a level, so we
only do them once per source object rather than once per link.

This allows LMDB joins to succeed with ~15,000 members in a group.
Previously LMDB would fail with the error:

 Failed to apply linked attribute change '(-30792) - MDB_MAP_FULL:
  Environment mapsize limit reached at ../lib/ldb/ldb_mdb/ldb_mdb.c:203'

Rewriting the same object ~15000 times seemed to completely fill up
the LMDB 8Gb buffer. Presumably this was because LMDB is 'copy on
write', so it was storing ~15,000 copies of the same object. Strangely,
we don't see this problem writing the backlinks (which this patch won't
have helped with at all, because that's modifying the target object).

Note uSNChanged was only being added to the msg once, so the code has
been modified to replace the usnChanged each time (i.e. remove it and
re-add it).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Nov  1 23:48:21 CET 2018 on sn-devel-144

- - - - -
4731c338 by Gary Lockyer at 2018-11-01T22:49:24Z
windbindd: reword error message

Reword the asprintf() out of memory message to make it clear where the
issue is.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
d6777a66 by Gary Lockyer at 2018-11-01T22:49:24Z
source4 smbd: Make the service_details structure constant.

Make the service_details structure a static const.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
99aea425 by Gary Lockyer at 2018-11-01T22:49:24Z
source4 smdb: Add a post fork hook to the service API

Add a post fork hook to the service API this will be called:

 - standard process model
   immediately after the task_init.

- single process model
  immediately after the task_init

- prefork process model, inhibit_pre_fork = true
  immediately after the task_init

- prefork process model, inhibit_pre_fork = false
  after each service worker has forked. It is not run on the service
  master process.

The post fork hook is not called in the standard model if a new process
is forked on a new connection. It is instead called immediately after
the task_init.

The task_init hook has been changed to return an error code. This ensures
the post_fork code is only run if the task_init code completed successfully.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
7366de33 by Gary Lockyer at 2018-11-01T22:49:25Z
source4 smbd kdc: allow the kdc to run in prefork

Modify the kdc to allow it to run in the prefork process model. The
task_init function has been split up and code moved into the post_fork
function.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
325e928d by Gary Lockyer at 2018-11-01T22:49:25Z
kdc: Update debug calls

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
28826ec4 by Gary Lockyer at 2018-11-02T02:47:57Z
WHATSNEW: KDC prefork support

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Fri Nov  2 03:47:57 CET 2018 on sn-devel-144

- - - - -
f17a77af by Douglas Bagnall at 2018-11-02T02:48:52Z
python dbcheck: don't use mutable default args

In this code

def f(a, b=[]):
    b.append(a)
    return b

all single argument calls to f() will affect the same copy of b.

In the controls case, controls=None has the same effect as
controls=[].

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1cf142c3 by Douglas Bagnall at 2018-11-02T02:48:52Z
selftesthelpers: fix py3 tests with extra_path

If a test was supplied with extra_path, a PYTHONPATH= env variable was
prepended to the args list, but the py3_compatible clause assumed the
first args element was /usr/bin/python.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
78f5b6e3 by Douglas Bagnall at 2018-11-02T02:48:52Z
s4/scripting/*: py3 compatible print

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
537a26d6 by Noel Power at 2018-11-02T05:56:24Z
tests/py/rodc_rwdc: Fix py2/py3 .next compat issues

Python 3 does not have .next(), which we rely on, change the
code slightly so it works in py2/py3 (using builtin next function)

Pair-programmed-with: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Nov  2 06:56:24 CET 2018 on sn-devel-144

- - - - -
831ee63f by Volker Lendecke at 2018-11-02T16:03:26Z
lib: Add error checks in dom_sid_string_buf

Also, avoid casts by using PRIxxx macros

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8b9d3622 by Volker Lendecke at 2018-11-02T19:11:11Z
lib: Add dom_sid_str_buf

This is modeled after server_id_str_buf, which as an API to me is easier to
use: I can rely on the compiler to get the buffer size right.

It is designed to violate README.Coding's "Make use of helper variables", but
as this API is simple enough and the output should never be a surprise at all,
I think that's worth it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Nov  2 20:11:11 CET 2018 on sn-devel-144

- - - - -
94ad5ee6 by Ralph Boehme at 2018-11-02T20:21:13Z
libcli: add smbXcli_req_endtime

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e7eec24d by Ralph Boehme at 2018-11-02T20:21:13Z
libcli: fill endtime if smbXcli_req_create() timeout is non-zero

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a6de555c by Ralph Boehme at 2018-11-02T20:21:13Z
s4:libcli/smb2: reapply request endtime

tevent_req_finish() removed a possible request timeout, make sure to
reinstall it. This happened when an interim SMB2 response was received.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
35f9bc06 by Ralph Boehme at 2018-11-02T20:21:14Z
vfs_delay_inject: implement pread_send and pwrite_send

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e37ff8c5 by Ralph Boehme at 2018-11-02T20:21:14Z
s4:torture/smb2/read: add test for cancelling SMB aio

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
07eb805c by Ralph Boehme at 2018-11-02T20:21:14Z
s3:smbd: fix SMB2 aio cancelling

As we currently don't attempt to cancel the internal aio request, we
must ignore the SMB2 cancel request and continue to process the SMB2
request, cf MS-SM2 3.3.5.16:

  If the target request is not successfully canceled, processing of the
  target request MUST continue and no response is sent to the cancel
  request.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3fa45900 by Ralph Boehme at 2018-11-02T20:21:14Z
s3:smbd: remove now unused check if fsp is NULL

This was used internally to mark an aio request as cancelled. As the aio
cancellation functionality has been removed, we can now also remove this
check.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
07d05202 by Volker Lendecke at 2018-11-02T20:21:14Z
nbt_server: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b9c590a6 by Volker Lendecke at 2018-11-02T20:21:14Z
rpc_server4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
46026053 by Volker Lendecke at 2018-11-02T20:21:14Z
dsdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4c1d1076 by Volker Lendecke at 2018-11-02T20:21:14Z
auth: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
71549537 by Volker Lendecke at 2018-11-02T20:21:15Z
idmap: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dbc9a1a7 by Volker Lendecke at 2018-11-02T20:21:15Z
net: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d0a56a55 by Volker Lendecke at 2018-11-02T20:21:15Z
audit_tests: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
602d247e by Volker Lendecke at 2018-11-02T20:21:15Z
auth4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
bd1e43f7 by Volker Lendecke at 2018-11-02T20:21:15Z
winbindd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
528170ef by Volker Lendecke at 2018-11-02T20:21:15Z
winbindd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ce18cd2a by Volker Lendecke at 2018-11-02T20:21:15Z
smbd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
039aef70 by Volker Lendecke at 2018-11-02T20:21:15Z
libsmb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
be6f45ca by Volker Lendecke at 2018-11-02T20:21:15Z
lib: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7963e409 by Volker Lendecke at 2018-11-02T20:21:16Z
passdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
74c7e3d1 by Volker Lendecke at 2018-11-02T20:21:16Z
audit_logging: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ab542ed2 by Volker Lendecke at 2018-11-02T20:21:16Z
auth: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7d958162 by Volker Lendecke at 2018-11-02T20:21:16Z
lib: Use dom_sid_str_buf

This is the one place where we have to do another strpcy. Many of the
sid_to_fstring calls should be replacable by dom_sid_str_buf, so this
will get less.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c2b9b574 by Volker Lendecke at 2018-11-02T23:23:31Z
lib: Make dom_sid_string_buf static

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Nov  3 00:23:31 CET 2018 on sn-devel-144

- - - - -
dc9bbbe4 by Christof Schmitt at 2018-11-03T02:01:25Z
smbtorture: Add test for DELETE_ON_CLOSE on files with READ_ONLY attribute

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13673

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
162a5257 by Christof Schmitt at 2018-11-03T02:01:25Z
smbd: Fix DELETE_ON_CLOSE behaviour on files with READ_ONLY attribute

MS-FSA states that a CREATE with FILE_DELETE_ON_CLOSE on an existing
file with READ_ONLY attribute has to return STATUS_CANNOT_DELETE. This
was missing in smbd as the check used the DOS attributes from the CREATE
instead of the DOS attributes on the existing file.

We need to handle the new file and existing file cases separately.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13673

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a8e79dec by Christof Schmitt at 2018-11-03T02:01:25Z
selftest: Add share to test "delete readonly" option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13673

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7dd3585f by Christof Schmitt at 2018-11-03T04:55:45Z
selftest: Run smb2.delete-on-close-perms also with "delete readonly = yes"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13673

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Sat Nov  3 05:55:45 CET 2018 on sn-devel-144

- - - - -
e3e84b0f by Tim Beale at 2018-11-04T22:55:15Z
traffic_replay: Add helper class for group assignments

Wrap up the group assignment calculations in a helper class. We're going
to tweak the internals a bit in subsequent patches, but the rest of the
code doesn't really need to know about these changes.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
18740ec0 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Split out random group membership generation logic

This doesn't change functionality at all. It just moves the probability
calculations out into separate functions.

We want to tweak the logic/implementation behind this code, but the
rest of assign_groups() doesn't really care how the underlying
probabilities are worked out, so long as it gets a suitably random
user/group membership each time round the loop.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
898e6b43 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Improve assign_groups() performance with large domains

When assigning 10,000 users to 15 groups each (on average),
assign_groups() would take over 30 seconds. This did not include any DB
operations whatsoever. This patch improves things, so that it takes less
than a second in the same situation.

The problem was the code was looping ~23 million times where the
'random.random() < probability * 10000' condition was not met. The
problem is individual group/user probabilities get lower as the number
of groups/users increases. And so with large numbers of users, most of
the time the calculated probability was very small and didn't meet the
threshold.

This patch changes it so we can select a user/group in one go, avoiding
the need to loop multiple times.

Basically we distribute the users (or groups) between 0.0 and 1.0, so
that each user has their own 'slice', and this slice is proporational to
their weighted probability. random.random() generates a value between
0.0 and 1.0, so we can use this to pick a 'slice' (or rather, we use
this as an index into the list, using .bisect()). Users/groups with
larger probabilities end up with larger slices, so are more likely to
get picked.

The end result is roughly the same distribution as before, although the
first 10 or so user/groups seem to get picked more frequently, so the
weighted-probability calculations may need tweaking some more.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fdd75407 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Change user distribution to use Pareto Distribution

The current probability we were assigning to users roughly approximates
the Pareto Distribution (with shape=1.0). This means the code now uses a
documented algorithm (i.e. explanation on Wikipedia). It also allows us
to vary the distribution by changing the shape parameter.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5ad7fc73 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Prevent users having 1000+ memberOf links

When adding 10,000 users, one user would end up in over 1000 groups.
With 100,000 users, it would be more like 10,000 groups. While it makes
sense to have groups with large numbers of users, having a single user
in 1000s of groups is probably less realistic.

This patch changes the shape of the Pareto distribution that we use to
assign users to groups. The aim is to cap users at belonging to at most
~500 groups. Increasing the shape of the Pareto distribution pushes the
user assignments so they're closer to the average, and the tail (with
users in lots of groups) is not so large).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a29ee3a7 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Re-organize assignments to be group-based

We can speed up writing the group memberships by adding multiple users
to a group in a single DB modify operation.

To do this, we first need to reorganize the assignments so instead
of being a set of tuples, it's a dictionary where key=group and
value=list-of-users-in-group.

add_users_to_groups() now iterates through the users/groups slightly
differently, but mostly it's just indentation changes. We haven't
changed the number of DB operations yet - we'll do that in the next
patch.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
32e58227 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Write group memberships once per group

Each user-group membership was being written to the DB in a single
operation. With large numbers of users (e.g. 10,000 in average 15 groups
each), this becomes a lot of operations (e.g. 150,000). This patch
reworks the code so that we write the memberships for a group in
one operation. E.g. instead of 150,000 DB operations, we might make
1,500. This makes writing the group memberships several times
faster.

Note that rthere is a performance vs memory tradeoff. When we hit
10,000+ members in a group, memory-usage in the underlying DB modify
operation becomes very inefficient/costly. So we avoid potential memory
usage problems by writing no more than 1,000 users to a group at once.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
71c66419 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Convert print() to logger.info()

Using logger is more helpful here because it includes timestamps, so we
can see how long things are taking. It's also more consistent with the
rest of the traffic_replay logging.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1906312c by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Improve user generation debug

When creating 1000s of users you currently get a lot of debug, but at
the same time you have no idea how far through creating the users you
actually are.

Instead of logging every single user account that's created, log every
50th (as well as how far through the overall generation we are).

Logger already includes timestamps, so we can remove generating the
timestamp diff manually. User creation is the slowest operation - adding
groups/memberships is much faster, so we don't need to log as
frequently.

Note that there is a usability trade-off on how frequently we log
depending on whether the user is using the slower (but more common)
method of going via LDAP, vs the much faster (but more obscure) method
of writing directly to sam.ldb with ldb:nosync=true. In my tests, we end
up logging every ~30-ish secs with LDAP, and every ~3 seconds with
direct file writes.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
be51b512 by Tim Beale at 2018-11-04T22:55:17Z
traffic_replay: Generate machine accounts as well as users

Currently the tool only generates the machine accounts needed for
traffic generation. However, this isn't realistic if we're trying to use
the tool to generate users to simulate a large network.

This patch generates machine accoutns along with the user accounts.
Note we assume there will be more computer accounts than users in a real
network (e.g. work laptops, servers, etc), so generate slightly more
computer accounts.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
3338a3e2 by Tim Beale at 2018-11-05T02:43:24Z
traffic: Machine accounts were generated as critical objects

Due to the userAccountControl flags we were specifying, the machine
accounts were all created as critical objects. When trying to populate
1000s of machine accounts in a DB, this makes replication unnecessarily
slow (because it has to replicate them all twice).

This patch changes it so when we're just creating machine accounts for
the purpose of populating a semi-realistic DB, we jsut use the default
WORKSTATION_TRUST_ACCOUNT flag.

Note that for the accounts used for traffic-replay, we apparently need
the existing flags in order for the DC to accept certain requests.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Mon Nov  5 03:43:24 CET 2018 on sn-devel-144

- - - - -
27df4f00 by Martin Schwenke at 2018-11-05T05:52:33Z
ctdb-recovery: Ban a node that causes recovery failure

... instead of applying banning credits.

There have been a couple of cases where recovery repeatedly takes just
over 2 minutes to fail.  Therefore, banning credits expire between
failures and a continuously problematic node is never banned,
resulting in endless recoveries.  This is because it takes 2
applications of banning credits before a node is banned, which
generally involves 2 recovery failures.

The recovery helper makes up to 3 attempts to recover each database
during a single run.  If a node causes 3 failures then this is really
equivalent to 3 recovery failures in the model that existed before the
recovery helper added retries.  In that case the node would have been
banned after 2 failures.

So, instead of applying banning credits to the "most failing" node,
simply ban it directly from the recovery helper.

If multiple nodes are causing recovery failures then this can cause a
node to be banned more quickly than it might otherwise have been, even
pre-recovery-helper.  However, 90 seconds (i.e. 3 failures) is a long
time to be in recovery, so banning earlier seems like the best
approach.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13670

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Nov  5 06:52:33 CET 2018 on sn-devel-144

- - - - -
0934fc14 by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: PY2/PY3 compat changes for __init__.py

Fixes.

1) sorting of xml.etree.ElementTree.Element, in PY2 sort
   seems to sort lists of these. In PY3 this no longer works.
   Choosing tag as the sort key for py3 so at least in python3
   there is a consistent sort (probably won't match how it is
   sorted in PY2 but nothing seems to depend on that)
2) md5 requires bytes
3) tostring returns bytes in PY3, adjust code for that

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
388bddf4 by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: PY3 file -> open

'file' no longer exists in PY3 replace with 'open'

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d40ef736 by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: PY3 open file non-binary mode for write_binary

Although this is unintuitive it's because we are writing unicode
not bytes (both in PY2 & PY3). using the 'b' mode causes an error in
PY3.

In PY3 we can define the encoding, but not in PY2.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
cf79e6ae by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: PY2/PY3 compat porting for gp_init.py

Fixes
1) use compat versions of ConfigParser and StringIO
2) fix sort list of XML Elements
3) open file needs to be opened in binary mode as write_pretty_xml
   routing uses BytesIO() object.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
df578e15 by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: Use csv.reader for parsing cvs files

The previous version here was using UnicodeReader which was
wrapping the UTF8Recoder class and passing that to csv.reader.
It looks like the intention was to read a bytestream in a
certain encoding and then reencode it to a different encoding.
And then UnicodeReader creates unicode from the newly encoded stream.
This is unnecssary, we know the encoding of the bytesstream and
codec.getreader will happily consume the bytstream and give back
unicode. The unicode can be fed directly into csv.writer.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
54e2bb70 by Noel Power at 2018-11-05T19:05:23Z
python/samab/gp_parse: remove unused code

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
19a459ba by Noel Power at 2018-11-05T19:05:24Z
python/samba/netcmd: misc PY2/PY3 compat changes for gpo.py

Fixes:
1) various ldb.bytes that should be displayed as strings in PY3
2) sorting of lists of xml Element in PY3
3) various 'open' need to be opened in binary mode (to accept binary
   data)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
16596842 by Noel Power at 2018-11-05T19:05:24Z
python/samba/gp_parse: Fix mulitple encode step with write_section

In python2 as far as I can see GptTmplInfParser.write_binary more
or less works by accident.

write_binary creates a writer for the 'utf8' codec, such a writer
should consume unicode and emit utf8 encoded bytes. This writer
is passed to each of the sections managed by GptTmplInfParser as
follows

    def write_binary(self, filename):
        with codecs.open(filename, 'wb+',
                         self.encoding) as f:
            for s in self.sections:
                self.sections[s].write_section(s, f)

And each section type itself is encoding its result to 'utf-16-le'
e.g.
    class UnicodeParam(AbstractParam):
         def write_section(self, header, fp):
            fp.write(u'[Unicode]\r\nUnicode=yes\r\n'.encode(self.encoding)

But this makes little sense, it seems like sections are encoded to one
encoding but the total file is supposed to be encoded as ut8??? Also
having an encoding per ParamType doesn't seem correct.

Bizarely in PY2 this works and it actually encodes the whole file as utf-16le
In PY3 you can't do this as the writer wants to deal with strings not bytes
(after the extra encode phase in 'write_section'.

So, changes here are to remove the unnecessary encoding in each 'write_section'
method, additionally in GptTmplInfParser.write_binary the
codecs.open call now uses the correct codec (e.g. 'utf-16-le') to write

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
6476ef58 by Noel Power at 2018-11-05T19:05:24Z
python/samba/tests/samba_tool: PY2/PY3 compat port for test

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fc047c2c by Noel Power at 2018-11-05T19:05:24Z
python/samba/gp_parse: PY2/PY3 Decode only when necessary

In python2 we decode str types in load_xml, in python3 these are
str class(s) which we cannot decode.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e355a6bc by Noel Power at 2018-11-05T22:04:48Z
s4/selftest: enable samba.tests.samba_tool.gpo for PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Mon Nov  5 23:04:48 CET 2018 on sn-devel-144

- - - - -
02d41fea by Tim Beale at 2018-11-06T02:39:11Z
join: Fix join large-DB timeout against Windows

The LDAP connection can also timeout when trying to join a Windows DC
with a very large database. However, in this case Windows gives a
slightly different error message (NT_STATUS_CONNECTION_RESET instead of
NT_STATUS_CONNECTION_DISCONNECTED).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13612

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c21a8ed5 by Tim Beale at 2018-11-06T02:39:11Z
libnet: Reset debug counters after replicating critical objects

Reset the debug counters once we have finished replicating a given
partition. This helps if we replicate the same partition immediately
afterward with different options.

This helps the DC join debug look less weird. Because it replicates the
critical objects first, and then the base partition, previously it
always ended up overcounting, e.g.

Partition[DC=addom,DC=samba,DC=example,DC=com] objects[314/218]
  linked_values[48/24]

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
63bfdb3c by Tim Beale at 2018-11-06T02:39:11Z
drs_utils: Fix some long lines

Tweak the code slightly to avoid some 80+ character lines.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2229f462 by Tim Beale at 2018-11-06T06:15:33Z
drs_utils: Avoid invalid dereference of v8 requests

req.more_flags only exists for v10 requests, so we throw an exception if
we try to dereference that field on a v8 (or v5) request. Unfortunately,
we were checking that we support v10 *after* we had tried to access the
more_flags. This patch fixes up the order of the checks.

This may be a problem trying to replicate with an older Windows DC
(pre-2008R2), and was reported on the samba mailing-list at one point:
https://lists.samba.org/archive/samba/2018-June/216541.html

Unfortunately this patch doesn't help the overall situation at all (the
join will fail because we can't resolve the link target and we can't use
GET_TGT). But it now gives you a more meaningful error, i.e.

  ERROR(runtime): uncaught exception - (8639, "Failed to process 'chunk'
    of DRS replicated objects: DOS code 0x000021bf"
instead of:
  ERROR(<type 'exceptions.AttributeError'>): uncaught exception -
    'drsuapi.DsGetNCChangesRequest8' object has no attribute 'more_flags'

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Tue Nov  6 07:15:33 CET 2018 on sn-devel-144

- - - - -
32c2ec8f by Martin Schwenke at 2018-11-06T06:16:13Z
ctdb-common: Allow path_socket() to use $CTDB_SOCKET

Use of CTDB_SOCKET is being generally removed.  However, this override
is being added to allow test code outside of ctdb/ to be able to
specify the socket, if desired.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d4a1f897 by Martin Schwenke at 2018-11-06T06:16:13Z
ctdb-tests: Use ctdb-path-like values for local daemons socket and PID file

However, don't use ctdb-path itself because some tests use nested
instances of onnode.  The outermost instance would set CTDB_SOCKET and
any inner instance would pick up that value, regardless of CTDB_BASE.

This is a temporary measure to avoid breaking testing while use of the
path functions is added to ctdbd and the ctdb tool.  When this is
complete these variables can be removed altogether because the code
will just depend on CTDB_TEST_MODE and CTDB_BASE.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
38566780 by Martin Schwenke at 2018-11-06T06:16:13Z
ctdb-tests: Use ctdb-path for fake_ctdbd directory setup

This needs to be done before any of the code changes are made,
including updating the ctdb tool.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
cc3aedd3 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tools: Use path_socket() in ctdb tool

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
cd021596 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Use path_socket() in test client tools

Just leak the memory allocated by path_socket().  This is only used in
short-lived test programs, so it isn't worth the hassle of plumbing a
talloc context through several layers to get here.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
5f478b7c by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-daemon: Use path functions for socket and PID file

Drop the use of ctdb_set_sockname() because it complicates the memory
allocation and this is the only place it is used.  Just assign to the
relevant pointer.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d75fa2c3 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-daemon: Drop unused function ctdb_set_socketname()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
82e589e3 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Drop setting of CTDB_SOCKET and CTDB_PIDFILE

The local daemons ssh stub doesn't need to do this because the ctdbd
and the ctdb tool now only need CTDB_TEST_MODE and CTDB_BASE for local
daemon tests.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
3b1e5977 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Drop incorrect comment, unused function

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d246b1da by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Use path_socket() in dummy client

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4f4a835c by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Remove export of CTDB_SOCKET

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
08469408 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: README updates

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
1bbc4fad by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tools: Detect unknown node number

If there aren't enough addresses in the list then the shift will
silently fail and the printed address will be the unshifted value of
$1, which is incorrect/unexpected.  So, sanity check the node number.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c84254d2 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-daemon: Avoid unnecessarily spamming the logs when in test mode

Logging the logging location to syslog can be useful on production
systems when the configuration goes unexpectedly missing.  However, in
test mode this just adds noise to the logs on the test system.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
9c41481f by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-daemon: Don't set log_to_stdout for become_daemon()

ctdbd logs to stderr in interactive mode, not stdout.  This way stdout
is always closed.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4e6bd424 by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-daemon: Improve documentation for -i option

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e952f031 by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-event: Never fork to become daemon in eventd

This stops ctdbd from being able to shut down eventd, since the PID it
records will be invalid.  There's no need for eventd to fork.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
44019b55 by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-event: Only run talloc report if CTDB_INTERACTIVE is set

This is only really wanted for interactive testing when logging to
stderr.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f420e8d5 by Jeremy Allison at 2018-11-06T06:16:15Z
lib: talloc: Mark talloc_autofree_context() as deprecated.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
698a9ef3 by Jeremy Allison at 2018-11-06T06:16:15Z
lib: ldb: Remove use of talloc_autofree_context().

Just use NULL in test case. talloc_autofree_context() is deprecated.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
eda1296d by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-tests: Add function wait_until_node_has_no_ips()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
5fdac517 by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-tests: Use wait_until_node_has_no_ips() in some tests

This strengthens those tests to ensure that released IPs aren't
replaced with others.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
ba86eacb by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-tests: Rationalise ctdb stop/continue/disable/enable simple tests

The "continue" and "enable" tests are just extensions of the "stop"
and "disable" tests, so drop the latter 2.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
43c26e1e by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-tests: Rationalise tunable simple tests

These 3 tests duplicate various checks and can easily be handled as a
single test.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
9ebcebe5 by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop useless "ctdb version" test

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
52056805 by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop use of confusing testfailures variable

Exit on first test failure instead of setting a variable.  The bizarre
logic in ctdb_test_exit() makes this worth dropping.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
6d9c89bf by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop setting of unused variable scriptname

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e733e4cb by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Ignore SIGPIPE during simple test cleanup

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0e9ead8f by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Start daemons in ctdb_test_init(), stop them in ctdb_test_exit()

This makes tests self-contained.  They can also now be individually
looped, if necessary.

Most tests (all but 1 complex, more than 50% of simple) restart the
daemons anyway, so this simplification is worth it.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
92337234 by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop cleanup_handler()

Running testsuite-specific code here isn&