[Pkg-samba-maint] [Git][samba-team/samba][experimental] 2951 commits: VERSION: Bump version up to 4.10.0pre1...

Mathieu Parent gitlab at salsa.debian.org
Fri Mar 22 06:38:07 GMT 2019


Mathieu Parent pushed to branch experimental at Debian Samba Team / samba


Commits:
37d056f4 by Karolin Seeger at 2018-07-12T08:11:01Z
VERSION: Bump version up to 4.10.0pre1...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
710ce1c3 by Karolin Seeger at 2018-07-12T08:15:44Z
WHATSNEW: Start release notes for Samba 4.10.

Signed-off-by: Karolin Seeger <kseeger at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
7b5a47b8 by Stefan Metzmacher at 2018-07-12T12:25:17Z
smbd: add [un]become_guest() helper functions

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
23319ef5 by Stefan Metzmacher at 2018-07-12T12:25:17Z
smbd: add smbd_impersonate_debug_create() helper

This will be used to implement no-op impersonation
for the create_conn_struct_as_root() case were we
don't really have other unrelated events in the loop
and only need a valid tevent wrapper context to avoid
double free on the raw event context on teardown.

This also adds useful debugging instead of being
a full no-op wrapper.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
5285966e by Stefan Metzmacher at 2018-07-12T12:25:17Z
smbd: add simple noop smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers

As a start these are just wrappers arround
smbd_impersonate_debug_create(), without any real impersonation.
But this will change shortly.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
0dcaa070 by Stefan Metzmacher at 2018-07-12T12:25:18Z
smbd: make use of smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers

For now they just add debugging, but that will change shortly.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
1b804f7a by Stefan Metzmacher at 2018-07-12T12:25:18Z
smbd: implement smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers

This makes sure we're doing the correct impersonation for async
requests, which is a requirement to start adding path based
async SMB_VFS calls.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
e37e41b3 by Stefan Metzmacher at 2018-07-12T12:25:18Z
smbd: avoid explicit change_to_user() in defer_rename_done() already done by impersonation

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
d0b1f96f by Stefan Metzmacher at 2018-07-12T12:25:18Z
smbd: remove unused change_to_root_user() from smbd_sig_hup_handler()

This is handled by using the root_ev_ctx in order to register
the signal event.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
c310647e by Ralph Boehme at 2018-07-12T12:25:18Z
smbd: remove unused change_to_root_user() from brl_timeout_fn()

This is handled by using the root_ev_ctx in order to register
the timer event.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
19e4a085 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: correctly handle pthreadpool_tevent_register_ev() failures

It returns errno values instead of setting 'errno'.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
5e723bc6 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: use unsigned for num_idle, num_threads and max_threads

These can't get negative.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
53a9f3ca by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: explicitly use max_thread=unlimited for pthreadpool_tevent_init() tests

Currently 0 also means unlimited, but that will change soon,
to force no thread and strict sync processing.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
65faef95 by Stefan Metzmacher at 2018-07-12T12:25:18Z
s3:messages: explicitly use max_thread=unlimited for pthreadpool_tevent_init() in messaging_dgm_init()

Currently 0 also means unlimited, but that will change soon,
to force no thread and strict sync processing.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
03830a32 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: consitently use unlock_res for pthread_mutex_unlock() in pthreadpool_add_job()

This makes further restructuring easier to implement and understand.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
e45d33e9 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: use strict sync processing only with max_threads=0

Otherwise it's an error if not at least one thread is possible.

This gives a much saner behaviour and doesn't end up with
unexpected sync processing.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
c9f54db1 by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: use talloc_zero() in tests_cmocka.c setup_pthreadpool_tevent()

This was found with valgrind.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f1dac71a by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: fix helgrind error in pthreadpool_free()

We need to pthread_mutex_lock/unlock the pool mutex
before we can destroy it.

The following test would trigger this.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
76474a6f by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: expand test_create() to check unlimited, sync and one thread pool

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
505d298e by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: add pthreadpool_max_threads() and pthreadpool_queued_jobs() helpers

These can be used to implement some kind of flow control in the caller.
E.g. unless pthreadpool_queued_jobs() is lower than
pthreadpool_max_threads() is good to prepare new jobs.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
e4dfd3da by Stefan Metzmacher at 2018-07-12T12:25:18Z
pthreadpool: add pthreadpool_tevent_max_threads() and pthreadpool_tevent_queued_jobs()

These can be used to implement some kind of flow control in the caller.
E.g. unless pthreadpool_tevent_queued_jobs() is lower than
pthreadpool_tevent_max_threads() is good to prepare new jobs.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
4e54543b by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: add pthreadpool_cancel_job()

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
59768416 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: don't process further jobs when shutting down

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f19552e2 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: split out a pthreadpool_stop() from pthreadpool_destroy()

This can be used in combination with pthreadpool_cancel_job() to
implement a multi step shutdown of the pool.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
cdbad904 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: let pthreadpool_tevent_job_send() fail with an invalid pool

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
245d684d by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: split out pthreadpool_tevent_job from pthreadpool_tevent_job_state

This makes it much easier to handle orphaned jobs,
we either wait for the immediate tevent to trigger
or we just keep leaking the memory.

The next commits will improve this further.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
791c0514 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: add pthreadpool_tevent_job_cancel()

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
fa070d90 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: make use of pthreadpool_stop() in pthreadpool_tevent_destructor()

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
25756425 by Stefan Metzmacher at 2018-07-12T12:25:19Z
pthreadpool: maintain a global list of orphaned pthreadpool_tevent_jobs

Instead of leaking the memory forever, we retry the cleanup,
if other pthreadpool_tevent_*() functions are used.

pthreadpool_tevent_cleanup_orphaned_jobs() could also be called
by external callers.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
3eee52b4 by Stefan Metzmacher at 2018-07-12T15:18:00Z
pthreadpool: allocate glue->tctx on glue as memory context.

This means it will go aways together with glue and thte event context.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Thu Jul 12 17:18:01 CEST 2018 on sn-devel-144

- - - - -
ce9ac51e by David Mulder at 2018-07-12T20:11:22Z
gpo: Fix asserts in gpo testing

These tests weren't using python's unit testing
asserts.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2494e338 by David Mulder at 2018-07-12T20:11:22Z
gpo: Disable python3 testing

The gpo module doesn't work in python3 yet,
causing this test to fail on python3.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13525

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4c7348e4 by David Mulder at 2018-07-12T20:11:22Z
python: Allow forced signing via smb.SMB()

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
57faf35c by David Mulder at 2018-07-12T20:11:22Z
gpo: Read GPO versions locally, not from sysvol

Non-kdc clients cannot read directly from the
sysvol, so we need to store the GPT.INI file
locally to read each gpo version.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2412c704 by David Mulder at 2018-07-12T20:11:23Z
gpo: Offline policy application via cache

Read policy files from the cache, rather than
the sysvol. This enables offline policy apply.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
46629f53 by David Mulder at 2018-07-12T20:11:23Z
param: Add python binding for lpcfg_state_path

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
387dc358 by David Mulder at 2018-07-12T20:11:23Z
gpo: add register_gp_extension for registering gp extensions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b4f09af2 by David Mulder at 2018-07-12T20:11:23Z
gpo: add unregister_gp_extension for unregistering gp extensions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4436b670 by David Mulder at 2018-07-12T20:11:23Z
gpo: add list_gp_extensions for listing registered gp extensions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fb6c250b by David Mulder at 2018-07-12T20:11:23Z
gpo: Tests for gp_ext register/unregister

Adds testing for the gp_ext register and
unregister functions, as well as testing
the list function.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
126b7892 by David Mulder at 2018-07-12T20:11:23Z
gpo: Dynamically load gp_exts

This loads Group Policy Client Side Extensions
similar to the way that they are loaded on a
Windows client. Extensions are installed to a
configuration file in the samba cache path where
they receive a unique GUID matched with the path
to the python gp_ext file. Classes which inherit
from the gp_ext class (as defined in gpclass.py)
will be dynamically loaded.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
de8b30a7 by David Mulder at 2018-07-12T20:11:23Z
gpo: Add user policy extensions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b05b6198 by David Mulder at 2018-07-12T20:11:23Z
gpo: Don't duplicate guids in the apply log

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f3358f04 by David Mulder at 2018-07-12T23:05:22Z
gpo: Specify samba module when importing from gpclass

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Jul 13 01:05:22 CEST 2018 on sn-devel-144

- - - - -
9b164637 by Joe Guo at 2018-07-12T23:12:24Z
samdb: use int for get and set methods instead of digit str

This will make the API work in a nature way.

Also, because of a defect in ldb API, code like `res[0]["maxPwdAge"][0]`
will return bytes even in Python3, which will cause trouble. By casting
the value to int, we avoid the str/bytes issue.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5b96e349 by Noel Power at 2018-07-12T23:12:24Z
python/samba/tests: Py2/Py3 allow import of ndr_(un)pack to work

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d5cd9af7 by Noel Power at 2018-07-12T23:12:24Z
python/samba/tests: Py2/Py3 port for hexdump

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7d435711 by Noel Power at 2018-07-12T23:12:24Z
python/samba/emulate: Fix some more missed exception tuple assignments

In python3 we need to change

    except LdbError as e:
-        (status, _) = e
to
    except LdbError as e:
+        (status, _) = e.args

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
337ae8cd by Noel Power at 2018-07-12T23:12:24Z
python/samba/tests: remove Py2 specific imports.

Remove some python2 specific import, probably this was due to
previous unavailability for some c-modules in python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7b170206 by Noel Power at 2018-07-12T23:12:24Z
lib/ldb: Implement a bytes derived object for attributes py2/py3

ldb attributes are either bytes (py3) or str (py2)

Some places in the code do str(res[0]['attribute'][0])
which results in
   'result' (py2)
  b'result' (py3)

or more commonly the attribute is used to construct a string e.g.
   "blah=" + res[0]['attribute'][0] + ",foo,bar=...."

giving
   "blah=result,foo,bar=...." (py2)
and very unhelpfully
   "blah=b'result',foo,bar=...." (py3)

lots of code already constructs various strings for passing to other
api using the above. To avoid many excessive
    res[0]['attribute'][0].decode('utf8')

code like 'res[0]['attribute'][0]'

will now return LdbBytes (a new object subclassing 'bytes') in py3
instead of bytes. This object has a custom '__str__' method which
attempts to return a string decoded to uft8. In Py2 this will behave as
it did previously (this is the safer option at the moment)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e915d18e by Noel Power at 2018-07-12T23:12:24Z
s4/torture/drs/python: Py2/Py2 fix tab/space also incorrect unicode usage

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
352eee57 by Noel Power at 2018-07-12T23:12:24Z
python/samba/netcmd: fix py2/py3 bytes usage for replace

base64.b64encode returns bytes in py3 make sure associated replace
uses 'b' for strings passed to replace

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
c46fd168 by Noel Power at 2018-07-12T23:12:24Z
python/samba/netcmd: Protect variable that can be None

In py3 None variable cannot be compared with '>' '<' etc operators

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
bbc5d314 by Noel Power at 2018-07-12T23:12:24Z
python/samba/netcmd: Fix relative module import

Part of future changes needed to enable samba4.drs.replica_sync_rodc
for PY3.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
883c3d8d by Noel Power at 2018-07-12T23:12:24Z
s4/librpc/ndr: allow GUID to accept unicode also

This needed since _GUID_string method change
(in source4/torture/drs/python/drs_base.py) which makes use use
a unicode guid at times now

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
2a2c2213 by Noel Power at 2018-07-12T23:12:25Z
s4/dsdb/tests/python: base64.b64encode returns bytes

adjust to unicode for py2/py3 compat needed as part of changes
to ensure samba4.ldap.password_settings will work with PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
343dc311 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs: ndr_upack needs bytes in py3 (samba4.drs.repl_move)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
06d3eac8 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: py2/py3 port map / ord usage

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
0e930dfb by Noel Power at 2018-07-12T23:12:25Z
python/samba: Add cmp_fn and cmp_to_key_fn functions for py2/py3

the cmp function and the cmp paramater (e.g. to sort functions)
no longer exist in python3.

cmp_fn is provides the missing functionality of the py2 cmp builtin
function.

cmp_to_key_fn allows the key paramater (e.g. for sort) to use the
old py2 cmp function for sorting. Note: the cmp_to_key is present in
since 2.7 (hence the inclusion of the source code for this function pre
that version)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>

- - - - -
77ee94c3 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: use cmp_fn and key=cmp_to_key_fn for py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
409ad5c0 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: use cmp_fn for cmp (for py2/py3 compat)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
08c9ce41 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: long is not used in py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fee61c57 by Noel Power at 2018-07-12T23:12:25Z
s4/torture/drs/python: xrange -> range for py2/py3 compat

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
f59a20eb by Noel Power at 2018-07-12T23:12:25Z
s4/torure/drs/python: Fix incorrect use of unicode which doesn't exist in py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1a9786fe by Noel Power at 2018-07-12T23:12:25Z
python/samba/tests: various py3 porting for ord/chr

various messages are lists of ints that need converting to str or bytes
depending on py2/py3, others are str/bytes that need modification and
are converted to lists or string char or ints for modificate (and then
reconstructed as str/bytes again)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d96d85a2 by Noel Power at 2018-07-12T23:12:25Z
python/samba/kcc: md5 needs to be passed bytes in py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
f8d97513 by Noel Power at 2018-07-13T02:01:59Z
python/samba/netcmd: Fix wrong exception referenced in code

post commit: 52729d35495db638c84caa8cc6f5ffdf0b670353 wrong
exception name was referenced.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Jul 13 04:01:59 CEST 2018 on sn-devel-144

- - - - -
c406a5be by Björn Jacke at 2018-07-13T09:44:02Z
docs: mention that the echo handler is for SMB1 only

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Karolin Seeger <kseeger at samba.org>

Autobuild-User(master): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(master): Fri Jul 13 11:44:02 CEST 2018 on sn-devel-144

- - - - -
1c8d1cce by Jeremy Allison at 2018-07-13T09:45:42Z
s3: torture: Test SMB1 cli_splice() fallback path when doing a non-full file splice.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13527

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>

- - - - -
c9656fd2 by Jeremy Allison at 2018-07-13T12:57:14Z
s3: libsmbclient: Fix cli_splice() fallback when reading less than a complete file.

We were always asking for SPLICE_BLOCK_SIZE even when the
remaining bytes we wanted were smaller than that. This works
when using cli_splice() on a complete file, as the cli_read()
terminated the read at the right place. We always have the
space to read SPLICE_BLOCK_SIZE bytes so this isn't an overflow.

Found by Bailey Berro <baileyberro at google.com>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13527

Signed-off-by: Bailey Berro <baileyberro at google.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>

Autobuild-User(master): David Disseldorp <ddiss at samba.org>
Autobuild-Date(master): Fri Jul 13 14:57:14 CEST 2018 on sn-devel-144

- - - - -
4e123c46 by Alexander Bokovoy at 2018-07-13T15:45:35Z
wafsamba/samba_abi: always hide ABI symbols which must be local

binutils 2.31 is going to change how shared libraries are linked, such
that they always provide their own local definitions of the _end, _edata
and _bss_start symbols.  This would all be fine, except for shared
libraries that export all symbols be default.  (Rather than just
exporting those symbols that form part of their API).

According to binutils developers, we should only export the symbols we
explicitly want to be used. We don't use this principle for all our
libraries and deliberately don't want to have ABI versioning control for
all of them, so the change I introduce here is to explicitly mark those
symbols that will always be added by default linker configuration with
binutils 2.31 as local. Right now these are '_end', '_edata', and
'__bss_start' symbols.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Jul 13 17:45:36 CEST 2018 on sn-devel-144

- - - - -
49d6c3f0 by David Disseldorp at 2018-07-13T22:14:13Z
s3: torture: adjust SMB1 cli_splice() test sizes

The test writes 20M and then splices just over 1M (13M is intended,
but there's a 1024*0124 typo). Fix the type and reduce the size of the
dataset to make it run faster - cli_splice works with 1M chunks
(SPLICE_BLOCK_SIZE), and the reproducer only requires that the splice
size is not chunk-aligned.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Jul 14 00:14:13 CEST 2018 on sn-devel-144

- - - - -
26fd7096 by Günther Deschner at 2018-07-14T01:09:00Z
s3-tldap: do not install test_tldap

Guenther

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13529

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Jul 14 03:09:00 CEST 2018 on sn-devel-144

- - - - -
8cb96438 by David Disseldorp at 2018-07-16T19:11:30Z
dbwrap: determine basename once instead of three times

Currently determined twice in the clear-if-first codepath and once in
the ctdb code path. Do it once at the top of the function.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Mon Jul 16 21:11:30 CEST 2018 on sn-devel-144

- - - - -
3761d42e by Ralph Boehme at 2018-07-17T11:33:06Z
tevent: fix CID 1437976 dereference before null check

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
ba991da4 by Ralph Boehme at 2018-07-17T14:21:21Z
tevent: fix CID 1437974 dereference after null check

Probably a copy/paste error from the tevent_debug() statement a few
lines above as at this place we want to pass main_ev directly to
tevent_debug() anyway.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Jul 17 16:21:21 CEST 2018 on sn-devel-144

- - - - -
72531974 by Ralph Boehme at 2018-07-17T22:04:10Z
s4: torture: run test_durable_v2_open_reopen2_lease() in a subdirectory

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13535

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e60e9368 by Ralph Boehme at 2018-07-17T22:04:10Z
s3: smbd: fix path check in smbd_smb2_create_durable_lease_check()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13535

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2e65a126 by Volker Lendecke at 2018-07-17T22:04:10Z
kcc: Fix the 32-bit build

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7b0c1f8f by Volker Lendecke at 2018-07-18T01:12:02Z
dsdb: Fix the 32-bit build

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Jul 18 03:12:02 CEST 2018 on sn-devel-144

- - - - -
da115efd by Martin Schwenke at 2018-07-18T09:51:14Z
ctdb-tests: Simplify pstree output in eventd unit tests

pstree truncates output when it exceeds a maximum width - the default
is 132 columns.  A couple of recent
commits (12fd8d7a5c5d14d403aac6cd9e318afcd0a8e159,
b23f3f996038626f618c5b5aa552686c1b852f44) lengthened the command
string in the output so that it is more likely to exceed this limit
and be truncated, as below:

==================================================
Running "cat /memdisk/autobuild/fl/b1851760/ctdb/ctdb/tests/var/eventd/debug_script.log"
--------------------------------------------------
Output (Exit status: 0):
--------------------------------------------------
02.enabled.scri,PID /memdisk/autobuild/fl/b1851760/ctdb/ctdb/tests/var/eventd/events/random/02.enabled.script ...
  `-sleep,PID 99
01.disabled          DISABLED
02.enabled           TIMEDOUT   DATETIME
  OUTPUT: Sleeping for 99 seconds
--------------------------------------------------
Required output (Exit status: 0):
--------------------------------------------------
02.enabled.scri,PID /memdisk/autobuild/fl/b1851760/ctdb/ctdb/tests/var/eventd/events/random/02.enabled.script verbosetimeout
  `-sleep,PID 99
01.disabled          DISABLED
02.enabled           TIMEDOUT   DATETIME
  OUTPUT: Sleeping for 99 seconds

FAILED

It isn't clear that the above example exceeds 132 characters, given
that the PID has been filtered into a fixed string, but it certainly
goes close.  Whether or not it is truncated probably depends on the
width of the PID in the unfiltered output.  This would explain why the
test flaps.

Avoid the output truncation by dropping the -a and -p options to
simplify the pstree output.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13531

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
359e5218 by Martin Schwenke at 2018-07-18T12:53:39Z
ctdb-tests: Loosen match against pstree output in simple test

As per previous commit, pstree can truncate output if it gets too
wide.  Instead of matching against the script's full path and
arguments, just match against the script name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13531

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Wed Jul 18 14:53:39 CEST 2018 on sn-devel-144

- - - - -
0d3aec18 by Gary Lockyer at 2018-07-19T23:47:20Z
dns wildcards: tests to confirm BUG 13536

DNS wildcard matching failing if more than one label to the left of the
wildcard. This commits adds tests to confirm the bug.

Wildcard entry: *.example.org
bar.example.com matches
foo.bar.example.com does not, but it it should.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
cef1b31c by Gary Lockyer at 2018-07-20T02:40:31Z
dns wildcards: fix BUG 13536

The current position in the dns name was not advanced past the '.'
character

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Jul 20 04:40:31 CEST 2018 on sn-devel-144

- - - - -
809967b3 by Jeremy Allison at 2018-07-20T10:25:16Z
s3: smbd: Fix Linux sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
16a58707 by Jeremy Allison at 2018-07-20T10:25:16Z
s3: smbd: Fix Solaris sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
d222caa4 by Jeremy Allison at 2018-07-20T10:25:16Z
s3: smbd: Fix HPUX sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
456e520a by Jeremy Allison at 2018-07-20T10:25:16Z
s3: smbd: Fix FreeBSD sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
582ce5d6 by Jeremy Allison at 2018-07-20T13:14:24Z
s3: smbd: Fix AIX sendfile() for SMB2. Ensure we don't spin on EAGAIN.

For SMB2 the socket is set non-blocking. Ensure sendfile()
calls complete if they return EAGAIN by saving the socket state,
setting it blocking, doing the sendfile until completion and then
restoring the socket state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13537

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Jul 20 15:14:24 CEST 2018 on sn-devel-144

- - - - -
2114768a by Alexander Bokovoy at 2018-07-23T14:24:10Z
s4-dns_server: Only build dns server Python code for AD DC

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
1faaeb00 by Alexander Bokovoy at 2018-07-23T14:24:10Z
s4-dsdb: only build dsdb Python modules for AD DC

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
6de9d878 by Alexander Bokovoy at 2018-07-23T17:36:56Z
python/samba/tests: make sure samba.tests can be imported without SamDB

We are using samba.tests Python module __init__.py file as a catch-all
for all types of helpers. Some of these helpers are only usable with
Samba AD DC targets.

When SamDB is not available in a non-Samba AD DC target, provide a
dummy replacement that simply returns None. This allows to complete
initialization for non-Samba AD DC target tests which do not use
connect_samdb() helper.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

Autobuild-User(master): Günther Deschner <gd at samba.org>
Autobuild-Date(master): Mon Jul 23 19:36:56 CEST 2018 on sn-devel-144

- - - - -
062b518c by Stefan Metzmacher at 2018-07-24T04:55:23Z
librpc: add binding handle support for [smb1]

This will be used to force smb1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
17b12a9b by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: split out smb_raw_negotiate_fill_transport()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
b7e99c25 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: add smbcli_transport_raw_init()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
87d73397 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: use talloc_zero() for struct smb_composite_connect in fetchfile.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
2b68f9b8 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: allow passing an already negotiated connection to smb_composite_connect()

It will just do the session setup and tree connect steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
ce2248c4 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: add smb2_transport_raw_init()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
5ad5b81b by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: split out smb2_connect_session_start()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
f20e607c by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: allow passing an already negotiated connection to smb2_connect_send()

It will just do the session setup and tree connect steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
ca000d89 by Stefan Metzmacher at 2018-07-24T04:55:23Z
s4:libcli: add fallback_to_anonymous to smb2_connect_send()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
5188454b by Stefan Metzmacher at 2018-07-24T04:55:24Z
s4:libcli: allow a fallback to NTLMSSP if SPNEGO is not supported locally

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
e4910f35 by Stefan Metzmacher at 2018-07-24T04:55:24Z
s4:libcli: add smb_connect_nego_{send,recv}()

This can be used to create a connection up to a negotiated
smbXcli_conn.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
70a9cf9c by Alexander Bokovoy at 2018-07-24T04:55:24Z
tests/auth_log: Permit SMB2 service description if empty binding is used for kerberos authentication

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>

- - - - -
802e43bf by Stefan Metzmacher at 2018-07-24T04:55:24Z
python/tests: use explicit "client ipc max protocol = NT1" for samba.tests.net_join_no_spnego

The tests rely on SMB1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
4422f738 by Stefan Metzmacher at 2018-07-24T04:55:24Z
s4:librpc: autonegotiate SMB1/2/3

Windows Server 1709 defaults to SMB2 and does not have SMB1 enabled.
When establishing trust, samba-tool does not specify SMB protocol
version and fail by default.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
6800077c by Stefan Metzmacher at 2018-07-24T04:55:24Z
s3:selftest: run rpc.lsa.lookupsids also with explicit [smb1] and [smb2]

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13308

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
c3907288 by Alexander Bokovoy at 2018-07-24T07:55:23Z
samba-tool trust: support discovery via netr_GetDcName

In case a remote DC does not support netr_DsRGetDCNameEx2(),
use netr_GetDcName() instead.

This should help with FreeIPA where embedded smbd runs as a domain
controller but does not implement full Active Directory compatibility.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13538

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Alexander Bokovoy <ab at samba.org>
Autobuild-Date(master): Tue Jul 24 09:55:23 CEST 2018 on sn-devel-144

- - - - -
9d31bbf9 by Stefan Metzmacher at 2018-07-24T15:38:25Z
tevent: use talloc_zero_size() for the private state in tevent_context_wrapper_create()

This is watch tevent_req_create() uses and what callers of
tevent_context_wrapper_create() would therefore also expect.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
26b35cb2 by Stefan Metzmacher at 2018-07-24T15:38:26Z
pthreadpool: make sure a pthreadpool is marked as stopped in child processes

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
617d9c87 by Stefan Metzmacher at 2018-07-24T15:38:26Z
pthreadpool: test pthreadpool_tevent_max_threads() returns the expected result

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
c51cae58 by Stefan Metzmacher at 2018-07-24T15:38:26Z
pthreadpool: replace assert_return_code(ret, 0); with assert_int_equal(ret, 0);

We need to assert the exact value!

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
2ebb5847 by Stefan Metzmacher at 2018-07-24T15:38:26Z
lib/replace: check for __thread support

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
6f8c1b67 by Stefan Metzmacher at 2018-07-24T15:38:26Z
third_party/*_wrapper/wscript: remove redundant configure checks

HAVE___THREAD and HAVE_DESTRUCTOR_ATTRIBUTE are already checked
as part of Samba.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
5fa5764f by Stefan Metzmacher at 2018-07-24T15:38:26Z
replace: add checks for atomic_thread_fence(memory_order_seq_cst) and add possible fallbacks

This implements a full memory barrier.
On ubuntu amd64 with results in an 'mfence' instruction.

This is required to syncronization between threads, where
there's typically only one write of a memory that should be
synced between all threads with the barrier.

Much more details can be found here:
https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/_005f_005fatomic-Builtins.html#g_t_005f_005fatomic-Builtins
https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/_005f_005fsync-Builtins.html#g_t_005f_005fsync-Builtins

The main one we use seems to be in C11 via stdatomic.h,
the oldest fallback is __sync_synchronize(), which is available
since 2005 in gcc.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9656b8d8 by Stefan Metzmacher at 2018-07-24T15:38:26Z
pthreadpool: add some lockless coordination between the main and job threads

In the direction from the main process to the job thread, we have:

- 'maycancel', which is set when tevent_req_cancel() is called,
- 'orphaned' is the job request, tevent_context or pthreadpool_tevent
  was talloc_free'ed.

The job function can consume these by using:

   /*
    * return true - if tevent_req_cancel() was called.
    */
   bool pthreadpool_tevent_current_job_canceled(void);

   /*
    * return true - if talloc_free() was called on the job request,
    * tevent_context or pthreadpool_tevent.
    */
   bool pthreadpool_tevent_current_job_orphaned(void);

   /*
    * return true if canceled and orphaned are both false.
    */
   bool pthreadpool_tevent_current_job_continue(void);

In the other direction we remember the following points
in the job execution:

- 'started'  - set when the job is picked up by a worker thread
- 'executed' - set once the job function returned.
- 'finished' - set when pthreadpool_tevent_job_signal() is entered
- 'dropped'  - set when pthreadpool_tevent_job_signal() leaves with orphaned
- 'signaled' - set when pthreadpool_tevent_job_signal() leaves normal

There're only one side writing each element,
either the main process or the job thread.

This means we can do the coordination with a full memory
barrier using atomic_thread_fence(memory_order_seq_cst).
lib/replace provides fallbacks if C11 stdatomic.h is not available.

A real pthreadpool requires pthread and atomic_thread_fence() (or an
replacement) to be available, otherwise we only have pthreadpool_sync.c.
But this should not make a real difference, as at least
__sync_synchronize() is availabe since 2005 in gcc.
We also require __thread which is available since 2002.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
30c97da7 by Stefan Metzmacher at 2018-07-24T15:38:26Z
s3:wscript: don't check for valgrind related headers twice

We already check them in lib/replace/wscript.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
66aaa22f by Stefan Metzmacher at 2018-07-24T15:38:27Z
lib/replace: also check for valgrind/helgrind.h

This will be used in lib/pthreadpool/pthreadpool_tevent.c
in order to avoid extected helgrind/drd warnings.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9b73fda9 by Stefan Metzmacher at 2018-07-24T15:38:27Z
pthreadpool: add helgrind magic to PTHREAD_TEVENT_JOB_THREAD_FENCE_*()

This avoids the expected helgrind/drd warnings on the job states which
are protected by the thread fence.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
aa9b64ec by Stefan Metzmacher at 2018-07-24T15:38:27Z
pthreadpool: maintain a list of job_states on each pthreadpool_tevent_glue

We should avoid traversing a linked list within a thread without holding
a mutex!

Using a mutex would be very tricky as we'll likely deadlock with
the mutexes at the raw pthreadpool layer.

So we use somekind of spinlock using atomic_thread_fence in order to
protect the access to job->state->glue->{tctx,ev} in
pthreadpool_tevent_job_signal().

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f23cac39 by Stefan Metzmacher at 2018-07-24T15:38:27Z
pthreadpool: add a comment about a further optimization in pthreadpool_tevent_job_destructor()

This seems to be a really rare race, it's likely that the immediate
event will still trigger and cleanup.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
40d15260 by Ralph Boehme at 2018-07-24T15:38:27Z
pthreadpool: test cancelling and freeing pending pthreadpool_tevent jobs/pools

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Ralph Boehme <slow at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>

- - - - -
7cb27238 by Ralph Boehme at 2018-07-24T15:38:27Z
configure: check for Linux specific unshare() with CLONE_FS

Note we still need some kind of runtime detection as
it can fail in some constraint container setups, which
reject the whole unshare() syscall instead of just the
once used for container features.

In case unshare(CLONE_FS) works, we can have a per thread
current working directory and use [f]chdir() safely in
worker threads.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
65e4742d by Ralph Boehme at 2018-07-24T15:38:27Z
pthreadpool: call unshare(CLONE_FS) if available

This paves the way for pthreadpool jobs that are path based.

Callers can use pthreadpool_per_thread_cwd() to check if
the current pool supports it.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
12a45ee1 by Stefan Metzmacher at 2018-07-24T15:38:27Z
pthreadpool: add pthreadpool_tevent_[current_job_]per_thread_cwd()

This can be used to check if worker threads run with
unshare(CLONE_FS).

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
fbafdc99 by Stefan Metzmacher at 2018-07-24T15:38:28Z
pthreadpool: add tests for pthreadpool_tevent_[current_job_]per_thread_cwd()

Note this currently this doesn't enforce the support for
unshare(CLONE_FS) as some contraint container environment
(e.g. docker) reject the whole unshare() system call.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
3c4cdb29 by Stefan Metzmacher at 2018-07-24T15:38:28Z
pthreadpool: add pthreadpool_restart_check[_monitor_{fd,drain}]()

This makes it possible to monitor the pthreadpool for exited worker
threads and may restart new threads from the main thread again.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f9745d8b by Stefan Metzmacher at 2018-07-24T15:38:28Z
pthreadpool: implement pthreadpool_tevent_wrapper_create() infrastructure

This can be used implement a generic per thread impersonation
for thread pools.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
fb6b6cf3 by Ralph Boehme at 2018-07-24T15:38:28Z
pthreadpool: test cancelling and freeing jobs of a wrapped pthreadpool_tevent

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
0d2eeb94 by Ralph Boehme at 2018-07-24T15:38:28Z
lib/util: rename USE_LINUX_THREAD_CREDENTIALS to HAVE_LINUX_THREAD_CREDENTIALS

The define reflects the results of a feature test, not a configure
option.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
10fc65b7 by Stefan Metzmacher at 2018-07-24T18:35:17Z
s3:util_sec: add a cache to set_thread_credentials()

Calling set_thread_credentials() with the same values,
skips syscalls the 2nd time.

We only do this if '__thread' is supported to provide
thread local storage.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Tue Jul 24 20:35:17 CEST 2018 on sn-devel-144

- - - - -
6a91ba4f by Volker Lendecke at 2018-07-24T18:36:49Z
lib: Remove an #include "includes.h"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a845e961 by Volker Lendecke at 2018-07-24T18:36:49Z
lib: Remove an #include "includes.h"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7daf5e62 by Volker Lendecke at 2018-07-24T18:36:50Z
lib: Fix prototype of srprs_str

Many callers use "-1" as the "len" argument. That's what ssize_t is for.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3c9b88ba by Volker Lendecke at 2018-07-24T18:36:50Z
nsswitch: Remove IRIX support

According to wikipedia, IRIX has seen the last patch update in August 2006. As
of now, www.sgi.com is unreachable. Probably this code has not been built in
years. If someone wants to revive it, it can be found in the git history.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d4c6b009 by Volker Lendecke at 2018-07-24T18:36:50Z
nsswitch: Make two functions static

nss_irix was the only external user

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
da179b1e by Volker Lendecke at 2018-07-24T18:36:50Z
nsswitch: Correct users of "ctx->is_privileged"

winbindd_context->is_privileged is a bool

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8a530433 by Volker Lendecke at 2018-07-24T18:36:50Z
winbind: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7d40f607 by Volker Lendecke at 2018-07-24T21:31:43Z
winbind: Move variable declarations close to their use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Jul 24 23:31:43 CEST 2018 on sn-devel-144

- - - - -
010bbe53 by Ralph Boehme at 2018-07-24T22:23:12Z
autobuild: add some basic tests for the all static build

This makes sure each module is at least loaded once
and registers itself as a module.

It means that the skel_opaque and skel_transparent vfs examples
are loaded.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
b294c7c6 by Ralph Boehme at 2018-07-24T22:23:12Z
examples/VFS/skel_opaque: add missing audit_file_fn

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
d1633532 by Ralph Boehme at 2018-07-24T22:23:12Z
examples/VFS/skel_opaque: add missing durable handle functions

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
f9db9aee by Stefan Metzmacher at 2018-07-24T22:23:12Z
examples/VFS/skel_opaque: call smb_vfs_assert_all_fns()

This template should always implement all calls.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
68b8e5a5 by Ralph Boehme at 2018-07-24T22:23:12Z
examples/VFS/skel_transparent: add missing audit_file_fn

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
829fdf10 by Ralph Boehme at 2018-07-24T22:23:12Z
examples/VFS/skel_transparent: add missing durable handle functions

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
42e99ec3 by Stefan Metzmacher at 2018-07-24T22:23:12Z
examples/VFS/skel_transparent: call smb_vfs_assert_all_fns()

This template should always include all calls.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
1bc92d10 by Stefan Metzmacher at 2018-07-24T22:23:12Z
vfs_default: call smb_vfs_assert_all_fns()

This module needs to implement every call.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
7d1de8bd by Ralph Boehme at 2018-07-24T22:23:12Z
s3: lib/xattr_tdb: fix listing xattrs

If there's no record in the xattr.tdb, dbwrap_fetch() will return
NT_STATUS_NOT_FOUND. That should not result in an error in callers of
xattr_tdb_load_attrs().

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
76c68bc2 by Ralph Boehme at 2018-07-24T22:23:13Z
s4: libcli/smb2: calculate correct credit charge for finds

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
580ff206 by Ralph Boehme at 2018-07-24T22:23:13Z
s3: libsmb: use smb2cli_conn_max_trans_size() in cli_smb2_list()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
27bb2cbc by Ralph Boehme at 2018-07-24T22:23:13Z
vfs_default: fix async fsync idle/busy time profiling

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
83f01b02 by Ralph Boehme at 2018-07-24T22:23:13Z
s3: vfs: add missing tevent_req_received() to SMB_VFS_PREAD_RECV()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
d769e9ea by Ralph Boehme at 2018-07-24T22:23:13Z
s3: vfs: add missing tevent_req_received() to SMB_VFS_PWRITE_RECV()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
55097f7d by Ralph Boehme at 2018-07-24T22:23:13Z
s3: vfs: add missing tevent_req_received() to SMB_VFS_FSYNC_RECV()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
b2ae22a3 by Ralph Boehme at 2018-07-24T22:23:13Z
s3: vfs: bump to version 39, Samba 4.9 will ship with that

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
1e78cb57 by Ralph Boehme at 2018-07-25T01:23:44Z
s3: vfs: bump to version 40, Samba 4.10 will ship with that

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Wed Jul 25 03:23:44 CEST 2018 on sn-devel-144

- - - - -
7f1de9ff by Jeremy Allison at 2018-07-25T01:24:39Z
s3: smbd: SGI IRIX is officially dead. Remove the kernel oplock code for IRIX.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>

- - - - -
e2ebfd8e by David Disseldorp at 2018-07-25T04:28:21Z
docs/kerneloplocks: drop Irix references

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Jul 25 06:28:21 CEST 2018 on sn-devel-144

- - - - -
79f494e5 by Gary Lockyer at 2018-07-25T04:29:50Z
json: Modify API to use return codes

Modify the auditing JSON API to return a response code, as the consensus
was that the existing error handling was aesthetically displeasing.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6f4f8c51 by Gary Lockyer at 2018-07-25T04:29:50Z
json: Add unit tests for error handling

Add cmocka unit tests to exercise the error handling in the JSON
routines.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a5172ee7 by Gary Lockyer at 2018-07-25T04:29:50Z
dsdb audit logging: remove HAVE_JANSSON from audit_log

This modules is ADDC only and JANSSON is required for the ADDC builds,
so the ifdef is no longer necessary.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
77fa3408 by Gary Lockyer at 2018-07-25T04:29:50Z
dsdb group auditing: remove HAVE_JANSSON from group_audit

This modules is ADDC only and JANSSON is required for the ADDC builds,
so the ifdef is no longer necessary.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6a5346f1 by Gary Lockyer at 2018-07-25T04:29:50Z
dsdb group_audit_test: Remove redundant mocking code

Remove a place holder test and unused mocking code.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a5e02f72 by Gary Lockyer at 2018-07-25T07:28:31Z
lib audit_logging: add _WARN_UNUSED_RESULT_

Have the compiler issue a warning when the return code from the API is
ignored.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Wed Jul 25 09:28:31 CEST 2018 on sn-devel-144

- - - - -
3fd1a41f by Ralph Boehme at 2018-07-25T15:49:05Z
pthreadpool: add a missing include

Reported-by: David Disseldorp <ddiss at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
5cb94ca0 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: only pass struct smbXsrv_client to smb1srv_tcon_disconnect_all()

That's all it needs.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
98487273 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: only pass struct smbXsrv_client to smbXsrv_session_logoff_all()

That's all it needs.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
8d4792e6 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: disconnect/destroy all connections before calling smbXsrv_session_logoff_all()

This means the pending requests are destroyed before tevent_context
impersonation wrapper are destroyed.

Otherwise, with a pending struct tevent_req that is a child of
client->xconn like this in exit_server_common():

conn[ipv4:127.0.0.11:40745] reason[NT_STATUS_CONNECTION_RESET] at ../source3/smbd/smb2_server.c:4015
full talloc report on 'struct smbXsrv_connection' (total   6085 bytes in  43 blocks)
    struct smbd_smb2_request       contains    648 bytes in   1 blocks (ref 0) 0x55ed41634740
    struct smbd_smb2_request       contains   3438 bytes in  32 blocks (ref 0) 0x55ed416331e0
        struct tevent_req              contains   1824 bytes in  20 blocks (ref 0) 0x55ed41635860
            struct smb_filename            contains    206 bytes in   2 blocks (ref 0) 0x55ed41635560
                lease_v2_complex2.dat          contains     22 bytes in   1 blocks (ref 0) 0x55ed4161b950
            struct smbd_smb2_create_state  contains   1386 bytes in  16 blocks (ref 0) 0x55ed41635a10
                struct deferred_open_record    contains    804 bytes in  12 blocks (ref 0) 0x55ed41633090
                    struct defer_open_state        contains    764 bytes in  11 blocks (ref 0) 0x55ed41634f10
                        struct tevent_req              contains    748 bytes in  10 blocks (ref 0) 0x55ed41636390
                            struct tevent_timer            contains     96 bytes in   1 blocks (ref 0) 0x55ed41636ad0
                            struct dbwrap_watched_watch_state contains    420 bytes in   7 blocks (ref 0) 0x55ed41636540
                                struct tevent_req              contains    296 bytes in   5 blocks (ref 0) 0x55ed41636700
                                    struct messaging_filtered_read_state contains     64 bytes in   3 blocks (ref 0) 0x55ed416368b0
                                        struct messaging_dgm_fde       contains      8 bytes in   2 blocks (ref 0) 0x55ed41636950
                                            reference to: struct messaging_dgm_fde_ev

we crash when freeing the xconn as the the tevent_req child has a
cleanup function that tries to access a (wrapped) tevent context that
was already destroyed via smb1srv_tcon_disconnect_all() for SMB1 or
smbXsrv_session_logoff_all() for SMB2 a few lines above.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
0e900d60 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: add missing DO_PROFILE_INC(disconnect) to smbd_server_connection_terminate_ex()

For multi channel connections we should increment the disconnect count
also if we're not closing the last channel.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
2be7518a by Ralph Boehme at 2018-07-25T15:49:06Z
smbd: rename sconn->pool to sconn->raw_thread_pool

This should in future not be used directly, we'll provide
wrapper pools, which will provide impersonation for
path based async calls.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0c972263 by Stefan Metzmacher at 2018-07-25T15:49:06Z
smbd: introduce sconn->sync_thread_pool

This just simulates a threadpool, but executes the
job functions inline (blocking) in the main thread.

This will be used to work arround some OS limitations,
e.g. if per thread credentials or per thread working directory
are not supported.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
1251a536 by Ralph Boehme at 2018-07-25T15:49:06Z
s3: vfs: add smb_vfs_ev_glue

This adds VFS helper functions and that work on a struct smb_vfs_ev_glue
object which bundles two event contexts and a few threadpools.

This will be used to streamline the use of impersonating wrappers
in the SMB_VFS.

Notice the verbose comments in source3/smbd/vfs.c.

This will allow us to introduce path based async operations
to the SMB_VFS layer.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
2dd95c1c by Ralph Boehme at 2018-07-25T15:49:06Z
s3: vfs: add user_vfs_evg to connection_struct

This will be used to in order to pass down the
impersonation magic from the SMB layer through
the SMB_VFS layer.

This includes the following options:

smbd:force sync user path safe threadpool
smbd:force sync user chdir safe threadpool
smbd:force sync root path safe threadpool
smbd:force sync root chdir safe threadpool

They can be used in order to test the non linux code
path on linux, once we get code that makes full use
of the new infrastructure.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
cd37badc by Ralph Boehme at 2018-07-25T15:49:07Z
vfs_aio_pthread: use event context and threadpool from user_vfs_evg

Or the root glue in case we're already root.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
bd79564a by Stefan Metzmacher at 2018-07-25T15:49:07Z
s3:modules: add vfs_not_implemented module

This provides helper functions, which can be used by other modules,
if they don't implement a specific function.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
53d78225 by Ralph Boehme at 2018-07-25T15:49:07Z
examples/VFS/skel_opaque: fix a likely a copy/paste error

This line was probably copied over from skel_transparent.c, remove it.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
f37f8cca by Ralph Boehme at 2018-07-25T15:49:07Z
examples/VFS/skel_opaque: make vfs_fn_pointers static

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
5ab0b4af by Stefan Metzmacher at 2018-07-25T18:44:12Z
examples/VFS/skel_transparent: make vfs_fn_pointers static

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Wed Jul 25 20:44:12 CEST 2018 on sn-devel-144

- - - - -
4e711d18 by Ralph Boehme at 2018-07-25T23:29:38Z
smbd: don't client->connections without checking client != NULL first in exit_server_common()

exit_server_common() can be called also in smbd processes without a
smbXsrv_client structure, e.g. the parent or some background tasks.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Thu Jul 26 01:29:38 CEST 2018 on sn-devel-144

- - - - -
fa9e6a50 by Volker Lendecke at 2018-07-26T20:44:24Z
idmap: Make pointer initialization explicit

Took me a few seconds to find this NULL initializer

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a3b42db9 by Volker Lendecke at 2018-07-26T20:44:25Z
popt: popt 1.16 needs -liconv

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3bff0494 by Volker Lendecke at 2018-07-26T20:44:25Z
smbd: Pass "share_mode_data" to share_entry_forall callback

Quite a bit of the contents have been passed explicitly anyway.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6ed119ec by Volker Lendecke at 2018-07-26T20:44:25Z
smbstatus: Use share_mode_data->leases

This is the only user of share_mode_entry->lease

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2dffcde4 by Volker Lendecke at 2018-07-26T23:42:31Z
smbd: Remove "share_mode_entry->lease"

smbstatus was the only user, and this could be solved by adapting
share_entry_forall.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Jul 27 01:42:31 CEST 2018 on sn-devel-144

- - - - -
5dd84bf5 by Martin Schwenke at 2018-07-27T03:45:20Z
ctdb-common: Fix compilation issue with strncpy()

When configured with --picky-developer and using -O3 with gcc 8.1:

../common/system_socket.c: In function ‘parse_ip_mask’:
../common/system_socket.c:229:2: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
  strncpy(s, str, len+1);
  ^~~~~~~~~~~~~~~~~~~~~~
../common/system_socket.c:223:8: note: length computed here
  len = strlen(str);
        ^~~~~~~~~~~

Use strlcpy() instead and check the result.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13545

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
3b56f200 by Martin Schwenke at 2018-07-27T03:45:20Z
ctdb-protocol: Fix compilation issue with strncpy()

When configured with --picky-developer and using -O3 with gcc 8.1:

../protocol/protocol_util.c: In function ‘ctdb_sock_addr_from_string’:
../protocol/protocol_util.c:282:2: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
  strncpy(s, str, len+1);
  ^~~~~~~~~~~~~~~~~~~~~~
../protocol/protocol_util.c:277:8: note: length computed here
  len = strlen(str);
        ^~~~~~~~~~~

Use strlcpy() instead and check the result.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13545

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4a1fb729 by Martin Schwenke at 2018-07-27T03:45:20Z
ctdb-protocol: Add function ctdb_sock_addr_mask_from_string()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
efaa7690 by Martin Schwenke at 2018-07-27T03:45:21Z
ctdb-tools: Switch to using ctdb_sock_addr_mask_from_string()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
b1fed7e1 by Martin Schwenke at 2018-07-27T03:45:21Z
ctdb-daemon: Switch to using ctdb_sock_addr_mask_from_string()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
77242e76 by Martin Schwenke at 2018-07-27T03:45:21Z
ctdb-common: Drop function parse_ip_mask() and supporting functions

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e7ef0c97 by Martin Schwenke at 2018-07-27T06:42:20Z
ctdb-common: Drop unused function mkdir_p_or_die()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Fri Jul 27 08:42:20 CEST 2018 on sn-devel-144

- - - - -
591d72f9 by Stefan Metzmacher at 2018-07-27T11:07:14Z
ldb_mdb: #ifdef EBADE as it is not portable

E.g. FreeBSD 11.2 doesn't have it.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
ff863f2d by Stefan Metzmacher at 2018-07-27T11:07:14Z
pthreadpool: we need to use pthreadpool_tevent_per_thread_cwd() on the callers pool

In pthreadpool_tevent_job_send() we remember if the job will be chdir
safe. It means we means we need to ask the callers pool when calling
pthreadpool_tevent_per_thread_cwd(), as the callers pool might
be a wrapper using pthreadpool_tevent_force_per_thread_cwd().

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
7e1f9c9d by Ralph Boehme at 2018-07-27T11:07:14Z
s3: vfs: add SMB_VFS_GETXATTRAT_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
c1c8a1f6 by Ralph Boehme at 2018-07-27T11:07:14Z
vfs_default: implement SMB_VFS_GETXATTRAT_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8a6013f6 by Ralph Boehme at 2018-07-27T11:07:14Z
vfs_xattr_tdb: implement SMB_VFS_GETXATTRAT_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
47d77432 by Ralph Boehme at 2018-07-27T11:07:14Z
s3: vfs: add SMB_VFS_GET_DOS_ATTRIBUTES_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0d1fb6a7 by Ralph Boehme at 2018-07-27T11:07:14Z
smbd: split out public parse_dos_attribute_blob() from get_ea_dos_attribute()

In preperation of adding an async version of get_ea_dos_attribute() that
will then call parse_dos_attribute_blob() too.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
1265b516 by Ralph Boehme at 2018-07-27T11:07:14Z
vfs_default: implement SMB_VFS_GET_DOS_ATTRIBUTES_SEND/RECV

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0feef168 by Ralph Boehme at 2018-07-27T11:07:14Z
smbd: factor out dosmode post processing

We apply some post processing to the dosmode returned from the VFS
function. Move this to a seperate function which will be reused in the
next commit in the async dos_mode_send/recv post processing.

Best viewed with: git show --histogram

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0fbeac03 by Ralph Boehme at 2018-07-27T11:07:14Z
smbd: add dos_mode_at_send/recv()

Signed-off-by: Ralph Boehme <slow at samba.org>

- - - - -
8de5018c by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: add "get_dosmode" argument to smbd_dirptr_lanman2_entry()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
2a2294bb by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: pass get_dosmode to smbd_dirptr_get_entry()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8954085a by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: pass get_dosmode to mode_fn in smbd_dirptr_get_entry()

This finally uses "get_dosmode" as passed in from the SMB2 layer, but
all callers still pass true, so no change in behaviour.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
bcfc830c by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: rework error exit in smbd_dirptr_lanman2_entry()

The next commit will add code that must be run if status is NT_STATUS_OK
or STATUS_MORE_ENTRIES.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
abef6ca7 by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: factor out smb2_query_directory_next_entry() from smbd_smb2_query_directory_send()

This paves the way for adding async functions into the enumeration loop.

Best viewed with: git show -w

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0e1d11ee by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: fix a long line in smb2_query_directory_next_entry()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
00a26ac9 by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: deal with fsp->aio_requests in close_directory()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8884036b by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: let smbd_dirptr_lanman2_entry return smb_fname

Note that smb_fname is relative to fsp, not conn!

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0736fdcd by Ralph Boehme at 2018-07-27T11:07:15Z
smbd: use async dos_mode_at_send in smbd_smb2_query_directory_send()

Finally: use the new dos_mode_at_send() in the directory enumeration
loop. This means that fetching the DOS attributes for directory entries
is done asynchronously with regard to the enumeration loop.

As the DOS attribute is typically read from an extended attribute in the
filesytem, this avoids sequentially blocking on IO. If the IO subsystem
is slow servicing these request, enabling async processing can result in
performance improvements.

A parametric option

  smbd:async dosmode = true | false (default: false)

can be used to enable the new async processing.

Simulating slow IO with usleep(5000) in the synchronous and asynchronous
versions of SMB_VFS_GET_DOS_ATTRIBUTES(), the results of enumerating a
directory with 10,000 files are:

    smbd:async dosmode = no:

        $ time bin/smbclient -U slow%x //localhost/test -c "ls dir\*" > /dev/null
        real    0m59.597s
        user    0m0.024s
        sys     0m0.012s

    smbd:async dosmode = yes:

        $ time bin/smbclient -U slow%x //localhost/test -c "ls dir\*" > /dev/null
        real    0m0.698s
        user    0m0.038s
        sys     0m0.025s

Performance gains in real world workloads depends on whether the actual
IO requests can be merged and parallelized by the kernel. Without such
wins at the IO layer, the async processing may even be slower then the
sync processing due to the additional overhead.

The following parameters can be used to adapt async processing behaviour
for specific workloads and systems:

        aio max threads = X (default: 100)
        smbd:max async dosmode = Y (default: "aio max threads" * 2)

By default we have at most twice the number of async requests in flight
as threads provided by the underlying threadpool. This ensures a worker
thread that finishes a job can directly pick up a new one without going
to sleep.

It may be advisable to reduce the number of threads to avoid scheduling
overhead while also increasing "smbd:max async dosmode".

Note that we disable async processing for certain VFS modules in the VFS
connect function to avoid the overhead of triggering the sync fallback
in dos_mode_at_send(). This is done for VFS modules that implement the
sync SMB_VFS_GET_DOS_ATTRIBUTES(), but not the async version (gpfs), and
for VFS modules that don't share a real filesystem where fchdir() can be
used (ceph, gluster). It is disabled for catia, because we realized that
the catia name translation macros used on
fsps (CATIA_FETCH_FSP_[PRE|POST]_NEXT) have a bug (#13547).

We use threadpool = smb_vfs_ev_glue_tp_chdir_safe() and then
pthreadpool_tevent_max_threads(threadpool) to get the number of maximum
worker threads which matches the pool used by the low level
SMB_VFS_GETXATTRAT_[SEND|RECV] implementation in vfs_default.

This is a terrible abstraction leak that should be removed in the future
by maybe making it possible to ask a VFS function which threadpool it
uses, internally suporting chaining so VFS function FOO that internally
uses BAR can forward the question to BAR.

On a hyphotetical system that had a getxattrat(dirfd, path, ...)
syscall and at the same time doesn't support per-thread current working
directories (eg FreeBSD doesn't have the latter) but has support for
per-thread-credentials, pthreadpool_tevent_max_threads() on the
tp_chdir_safe threadpool returns 1.

So when hooking the hyphotetical getxattrat() into the async
SMB_VFS_GETXATTRAT_[SEND|RECV] implementation in an VFS module, the
implementation could use the tp_path_safe threadpool, but the SMB2
layer would use the wrong threadpool in the call to
pthreadpool_tevent_max_threads(), resulting in no parallelism.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0b32efcd by Ralph Boehme at 2018-07-27T11:07:15Z
s4: torture: test closing dir handle with in-flight find

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
cbb7eb1f by Ralph Boehme at 2018-07-27T11:07:15Z
selftest: set "smbd:async dosmode = no" in the vfs_aio_pthread share

This just explicitly sets the current default, to ensure the tests that
use this share always use the same "smbd:async dosmode" setting even if
the default changes in the future.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
409d462f by Ralph Boehme at 2018-07-27T14:04:02Z
selftest: run smbtorture3 SMB2-BASIC tests against additional shares

This runs the smbtorture3 SMB2-BASIC and smb2.compound_find tests against shares
with "smbd:async dosmode" enabled.

On the vfs_aio_pthread_async_dosmode_force_sync* shares we
force a sync threadpool which ensures we test behaviour on systems that
don't support unshare(CLONE_FS) and also don't support
per-thread-credentials. This simulates the code path of non linux
systems. And makes sure that we don't regress there.

We also test with xattr_tdb and without.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Ralph Boehme <slow at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Fri Jul 27 16:04:02 CEST 2018 on sn-devel-144

- - - - -
100fe9e9 by Amitay Isaacs at 2018-07-27T22:25:29Z
popt: Check for headers only if building in-tree version

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
156ac244 by Amitay Isaacs at 2018-07-27T22:25:29Z
popt: Add check for iconv library

On glibc based systems, there is no separate iconv library.  Adding a
dependency without checking for the library breaks build on glibc based
systems.

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2ba5fb20 by Andrew Bartlett at 2018-07-28T01:39:48Z
autobuild: Test with and without building bundled popt

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Jul 28 03:39:48 CEST 2018 on sn-devel-144

- - - - -
73298ac8 by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tools: Improve portability by not using /bin/bash directly

FreeBSD and others do not have /bin/bash, so use "/usr/bin/env bash"
for better flexibility.

There are still many integration tests that use /bin/bash but this at
least lets FreeBSD start running tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
dd9d8a20 by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Improve portability by not using /bin/bash directly

FreeBSD and others do not have /bin/bash, so use "/usr/bin/env bash"
for better flexibility.

There are still many integration tests that use /bin/bash but this at
least lets FreeBSD start running tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4a39bc4a by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tools: Avoid use of non-portable getopt in onnode

getopt is being used with non-portable options.  Use simpler,
POSIX-compliant getopts instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
56ffca3e by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Avoid use of non-portable getopt in run_tests.sh

getopt is being used with non-portable options.  Use simpler,
POSIX-compliant getopts instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
896c77df by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Avoid use of non-portable getopt in stubs

getopt is being used with non-portable options.  In most cases use
simpler, POSIX-compliant getopts instead.

In the case of the ctdb test stub command, options can appear after
other arguments, so this requires an additional nested loop.

In the case of smnotify, there are no short options, so handle the
long options manually.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2f2c35a1 by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Improve portability by not using mktemp --tmpdir option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f13824b2 by Martin Schwenke at 2018-07-28T01:50:10Z
ctdb-tests: Switch some test stubs to use /bin/sh

They don't use any bash features.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
af8c31ea by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-tests: Add errno matching utility

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
e8a1b3db by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-tests: Add required_error() to match on error codes

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
a42a7232 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-common: Switch to ETIMEDOUT from ETIME

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c8756ec1 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-event: Switch to ETIMEDOUT instead of ETIME

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
b886a95e by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-daemon: Switch to using ETIMEDOUT instead of ETIME

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
e1236a85 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-client: Switch to ETIMEDOUT instead of ETIME

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
b7dbe9f3 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-tests: Add ps output filter for freebsd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
b0028dd5 by Amitay Isaacs at 2018-07-28T01:50:10Z
ctdb-tests: Add signal code matching utility

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
23952c91 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Use sigcode to match signals

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
0273171c by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Porting tests should ignore unsupported features

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c7041b0f by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-common: Add line based I/O

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c9b42d27 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-protocol: Avoid fgets in ctdb_connection_list_read

C library buffering API can behave in unexpected fashion if underlying
fd for stdin, stdout or stderr is closed and re-opened.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
3bf753e8 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-common: Add fd argument to ctdb_connection_list_read()

This makes testing easier.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
4152e98c by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Do not try to match pstree output in eventd tests

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
68542dbb by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Simplify pattern matching for ctime output

On freebsd, sed does not accept multiple pattern strings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
96d5c7de by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-scripts: date "+%N" is non-portable

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
07844c2e by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Use portable wc -c instead of stat -c "%s"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
22c3078c by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Replace md5sum with posix cksum

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
65cc36f2 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Use errcode to translate ETIMEDOUT

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
6f5ed2b8 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Fix a typo

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
3047202c by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-tests: Strip all spaces from od output

On freebsd, there are trailing spaces in od output.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
be43e080 by Amitay Isaacs at 2018-07-28T01:50:11Z
ctdb-common: Fix the TCP packet length check

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13520

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
a44e6987 by Martin Schwenke at 2018-07-28T05:26:24Z
ctdb-docs: Replace obsolete reference to CTDB_DEBUG_HUNG_SCRIPT option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13546

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Sat Jul 28 07:26:24 CEST 2018 on sn-devel-144

- - - - -
56e248de by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-event: Fix "ctdb event status" usage message

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a7a4ee43 by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-common: Factor out basic script abstraction

Provides for listing of scripts and chmod enable/disable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
82e62488 by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-common: Use script abstraction in run_event

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
295826f1 by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-event: Change event-tool script enable/disable to chmod file directly

They no longer go over the socket to eventd to enable and disable
scripts.  Use the event script abstraction to chmod them directly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
5017325c by Martin Schwenke at 2018-07-28T15:14:11Z
ctdb-event: Implement event tool "script list" command

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e3ce1a2d by Martin Schwenke at 2018-07-28T18:03:52Z
ctdb-docs: Update documentation for "ctdb event" command

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13551

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Sat Jul 28 20:03:52 CEST 2018 on sn-devel-144

- - - - -
d881f0c8 by Justin Stephenson at 2018-07-30T05:34:11Z
s3:libads: Add net ads leave keep-account option

Add the ability to leave the domain with --keep-account argument to avoid
removal of the host machine account.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13498

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
8025467b by Justin Stephenson at 2018-07-30T08:22:59Z
s3:libads: Add net ads keep-account test

Add test for the new --keep-account net ads leave operation

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13498

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Jul 30 10:22:59 CEST 2018 on sn-devel-144

- - - - -
4fcbaae5 by Martin Schwenke at 2018-07-30T12:30:06Z
ctdb-doc: Provide an example script for migrating old configuration

Include an example ctdbd.conf-style file for testing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13550

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Mon Jul 30 14:30:06 CEST 2018 on sn-devel-144

- - - - -
1c8ea099 by Gary Lockyer at 2018-07-30T12:31:52Z
lib ldb: Rename functions to ldb_kv

Rename the ldb key value functions from ltdb_* to ldb_kv_*. The renaming
is preparation for the separation of the tdb specific code from the key
value code.  This work is a follow on from the addition of the lmdb
backend.

Note that the next commit tidies up the code formatting.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9e629466 by Gary Lockyer at 2018-07-30T12:31:52Z
lib ldb: fix formatting of ldb_kv rename.

Clean up the code format after the rename in the previous commit.
Hopefully doing a rename commit followed by a reformat commit makes the
code easier to review.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e8aa764e by Gary Lockyer at 2018-07-30T12:31:52Z
lib ldb: rename struct ltdb_reindex_context

Rename struct ltdb_reindex_context to ldb_kv_reindex_context, as this is
a key value level structure and not a tdb specific structure.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e969de07 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: reformat ltdb_reindex_context rename

Fix up the formatting after the rename of ltdb_reindex_context to
ldb_kv_reindex_context.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c782d710 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename ltdb_context to ldb_kv_context

Rename ltdb_context to ldb_kv_context as it is a key value level
structure and not tdb specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
17a84d12 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename ltdb_req_spy to ldb_kv_req_spy

Rename ltdb_req_spy to ldb_kv_req_spy, as it is key value level and not
tdb specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
641b38e4 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: format rename of ltdb_req_spy

Fix up the code formatting after the rename of ltdb_req_spy to
ldb_kv_req_spy

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e0186d1f by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename ltdb_private to ldb_kv_private

Rename ltdb_private to ldb_kv_private as it contains key value operation
context.

Note there is still some tdb specific context that can be refactored into a
separate structure along the lines of the lmdb context.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b8c9c305 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: format rename ldb_kv_private

Tidy up the code format after the rename of ltdb_private to
ldb_kv_private

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
72724f75 by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename ltdb_cache to ldb_kv_cache

Rename ltdb_cache to ldb_kv_cache as it's key value level and not tdb
specific

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f6d5cf5e by Gary Lockyer at 2018-07-30T12:31:53Z
lib ldb: rename tdb_key_ctx to key_ctx

Rename tdb_key_ctx to key_ctx, as it's key value level and not tdb
specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d3bfd374 by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: rename ltdb_idxptr to ldb_kv_idxptr

Rename ltdb_idxptr to ldb_kv_idxptr as it's key value level and not tdb
specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9191d3ba by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: remove unused function prototypes

Remove unused function prototypes

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
67c05540 by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: rename ltdb_parse_data_unpack_ctx

Rename ltdb_parse_data_unpack_ctx to ldb_kv_parse_data_unpack_ctx, as
it's a key value level structure and not ltdb specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
19be0be2 by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: move key value code to lib/ldb/ldb_key_value

Move the key value code to a separate subdirectory.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f2d5c2c5 by Gary Lockyer at 2018-07-30T12:31:54Z
lib ldb: rename LTDB_* constants to LDB_KV_*

Rename all the LTDB_* constants to LDB_KV_* as they are key value level
constants and not tdb specific.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c891df42 by Gary Lockyer at 2018-07-30T15:23:22Z
lib ldb key value: convert TDB_DATA structs to ldb_val

Convert the key value functions to use ldb_val instead of TDB_DATA.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Jul 30 17:23:22 CEST 2018 on sn-devel-144

- - - - -
80c9219d by Anoop C S at 2018-07-31T02:23:47Z
s3/locking: Fix assertion check on lock reference count

lock_ref_count will always hold the old value prior to change. Thus it
would mean that if lock_ref_count is 0 the new value is already -1 which
is not expected here. Therefore it is better to make sure that it is
always greater than 0 rather than >= 0.

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Jul 31 04:23:47 CEST 2018 on sn-devel-144

- - - - -
6da0d68f by Stefan Metzmacher at 2018-07-31T09:26:16Z
pthreadpool: ignore the return value of poll(NULL, 0UL, 1)

Otherwise Coverity reports this:

CID 1438160:    (CHECKED_RETURN)
Calling "poll(NULL, 0UL, 1)" without checking return value. This
library function may fail and return an error code.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
f68b5ee7 by Stefan Metzmacher at 2018-07-31T09:26:16Z
pthreadpool: reset monitor_fd after calling tevent_fd_set_auto_close()

This tries to convince Coverity that we don't have a resource leak:

CID 1438157:    (RESOURCE_LEAK)
Handle variable "monitor_fd" going out of scope leaks the handle.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
8dac16e8 by Stefan Metzmacher at 2018-07-31T12:20:49Z
smb2_query_directory: make 'return true' explicit in smb2_query_directory_next_entry()

'return req' should do the same as 'return true' for a bool function,
it's implicitly expanded as 'return (req!=NULL)?true:false.

There's no point in that as 'req' is always a valid pointer.

This was most likely just a copy and paste bug.

So we make this explicit now and avoid that Coverity reports this:

CID 1438158:  Null pointer dereferences  (REVERSE_INULL)
Null-checking "req" suggests that it may be null, but it has already
been dereferenced on all paths leading to the check.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Jul 31 14:20:49 CEST 2018 on sn-devel-144

- - - - -
2bfb9b40 by Noel Power at 2018-07-31T14:56:24Z
s3/lib: Fix misleading typo in debug message

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b9340824 by Noel Power at 2018-07-31T14:56:24Z
s3/libsmb: Avoid potential smbpanic calling parse_user_quota_list.

Calling parse_user_quota_list with a NULL buffer can cause a panic, while
this shouldn't happen, I managed to trigger this with an early implementation
of SMB2 quota support in smbd which didn't pass back NT_STATUS_NO_MORE_ENTRIES
when handling a SMB2_0_INFO_QUOTA GETINFO message.
OTHOH the Windows client handled the same situation gracefully.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1033dd9e by Noel Power at 2018-07-31T14:56:24Z
s3/smbd: Don't stat when doing a quota operation (as it's a fake file)

calling SMB_VFS_STAT on the quota fake file fails and caused
FS_INFO/FileFsControlInfo request to error out early, in turn stopped a
Win8.1 client from proceeding with quota queries.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fd7c6f9b by Noel Power at 2018-07-31T14:56:25Z
librpc/idl Add some query [getset]info quota related structures

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
07e91132 by Noel Power at 2018-07-31T14:56:25Z
s3/libsmb: adjust smb1 cli code to use idl structs and ndr push/pull funcs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dd21b4fc by Noel Power at 2018-07-31T14:56:25Z
s3/libsmb: adjust smb2 code for new idl structs & generated ndr push/pull funcs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b8802e27 by Noel Power at 2018-07-31T14:56:25Z
s3/smbd: adjust smb1 server to use idl structs and generated ndr push/pull funcs

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
90703841 by Noel Power at 2018-07-31T14:56:25Z
s3/smbd: smb2 server implementation for query get/set info.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
90da2e59 by Noel Power at 2018-07-31T14:56:25Z
s3/script/test: modify existing smbcquota test to use SMB2 in addition to SMB1.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
801c1856 by Noel Power at 2018-07-31T14:56:25Z
s3/script/tests: Add simple (smb1 & smb2) get/set/list tests for smbcquotas

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
705086d8 by Noel Power at 2018-07-31T14:56:25Z
s3/smbd: allow set quota for non root user (when built with --enable-selftest)

Currently it appears you need to be root to set quotas, for test purposes
this requirement needs to be relaxed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e053aad4 by Noel Power at 2018-07-31T17:45:59Z
s3/utils: fix regression where specifying -Unetbios/root works

Usually you need to be root on a linux server to modify quotas. Even
with a linux server joined to a windows AD you could always log in as
local root with smbcquotas. However in recent builds this has changed.
This patch fixes this

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13553

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Tue Jul 31 19:45:59 CEST 2018 on sn-devel-144

- - - - -
33d012c3 by Volker Lendecke at 2018-08-03T06:24:06Z
ctdb: Fix a cut&paste error

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13554

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f7b2e5ee by Amitay Isaacs at 2018-08-03T09:14:01Z
ctdb-eventd: Fix CID 1438155

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13554

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Fri Aug  3 11:14:01 CEST 2018 on sn-devel-144

- - - - -
a3d248f2 by Oleksandr Natalenko at 2018-08-03T15:28:52Z
systemd: Only start smb when network interfaces are up

For smb, if the smb.conf contains explicit bindings to the network
interfaces, the service must wait till network interfaces are up,
otherwise the service won't be operational.

The 0e571054a61e commit and the BZ 13184 have fixed this for nmb and
samba, so do exactly the same here, for smb.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13559

Signed-off-by: Oleksandr Natalenko <oleksandr at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Aug  3 17:28:52 CEST 2018 on sn-devel-144

- - - - -
97702ffc by Justin Stephenson at 2018-08-06T00:46:16Z
Add net lookup options

Add missing net lookup options to net man page

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jim McDonough <jmcd at samba.org>

- - - - -
dea788e5 by Gary Lockyer at 2018-08-06T03:36:42Z
dns scavenging: Add extra tests for custom filter

Add extra tests for the custom ldb filter used by the dns scavenging
code.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Aug  6 05:36:43 CEST 2018 on sn-devel-144

- - - - -
8f83933f by Aaron Haslett at 2018-08-06T03:37:42Z
tdb: adding readonly locks mode to tdbbackup tool

The netcmd 'domain backup offline' command will use the tdbbackup tool but
require readonly locking of tdb databases, otherwise all database access would
be blocked during a backup.  This patch adds the option.  A backup script
should use this tool with the readonly locks option after taking a transaction
lock on the target database.

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
04217372 by Aaron Haslett at 2018-08-06T03:37:42Z
tdb: test for readonly locks mode on tdbbackup command

Simple bash test for readonly locks on tdbbackup:
1. Running tdbbackup on a database with and without readonly locks enabled.
2. Dump both backups and original.
3. Check all three dumps match.

A binary sample_tdb.tdb file is included for the test because the existing
sample tdbs in lib/tdb/test are either corrupt or empty.

Signed-off-by: Aaron Haslett <aaron.haslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4f532cc1 by Tim Beale at 2018-08-06T03:37:42Z
netcmd: Improve domain backup targetdir checks

+ Added check that specified targetdir is actually a directory (if it
exists)
+ Deleted a redundant 'Creating targetdir' check that would never be hit
+ Move code into a separate function so we can reuse it for offline
backups (which take a different set of parameters, but still have a
targetdir)

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
f17d2012 by Aaron Haslett at 2018-08-06T03:37:42Z
netcmd: domain backup offline command

Unlike the existing 'domain backup online' command, this command allows an
admin to back up a local samba installation using the filesystem and the
tdbbackup tool instead of using remote protocols.  It replaces samba_backup
as that tool does not handle sam.ldb and secrets.ldb correctly.  Those two
databases need to have transactions started on them before their downstream
ldb and tdb files are backed up.

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
2104818e by Tim Beale at 2018-08-06T03:37:43Z
tests: New offline backup tests with tweaks to old online classes

Offline backups have a slightly different syntax, as they don't take the
server or user-creds parameters. In the untar case, the offline backup
will actually have the secrets present, so making asserting on this
more flexible.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
2800611d by Aaron Haslett at 2018-08-06T03:37:43Z
netcmd: domain backup offline command - offline test with ldapcmp

This test checks that when you do an offline backup and restore or untar it,
the restored database is the same as the original.  Test is repeated for
'mdb' and 'tdb' database backends.

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
eb4161d7 by Aaron Haslett at 2018-08-06T06:45:19Z
selftest: offline backup restore target

This is a selftest target built from a restored offline backup.
Other backup routines are modified to remove the assumption that every backup
requires server and credentials arguments, since offline backup doesn't
want them.  Also, prepare_dc_testenv now returns the generated ctx so we can
run or re-run routines that require it later.

Signed-off-by: Aaron Haslett <aaron.haslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Mon Aug  6 08:45:19 CEST 2018 on sn-devel-144

- - - - -
d76c7b20 by Richard Sharpe at 2018-08-06T06:46:41Z
Minor, really small, documentation fix.

Signed-off-by: Richard Sharpe <realrichardsharpe at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c6b95c36 by Swen Schillig at 2018-08-06T09:37:32Z
ctdb: remove queue destructor as it isn't needed anymore

After

commit e097b7f8ff1a9992de1d11474dac4969e30cd679
Author: David Disseldorp <ddiss at suse.de>
Date:   Sun Jul 31 03:14:54 2011 +0200

    io: Make queue_io_read() safe for reentry

the destructor has no purpose anymore, therfore, remove it.

Signed-off-by: Swen Schillig <swen at vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: David Disseldorp <ddiss at samba.org>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Mon Aug  6 11:37:32 CEST 2018 on sn-devel-144

- - - - -
0530cccc by Andreas Schneider at 2018-08-06T23:49:34Z
s3:waf: Install eventlogadm to /usr/sbin

The eventlogadm binary needs write access to the registry which, by
default, is only possible as root.

https://bugzilla.samba.org/show_bug.cgi?id=13561

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Aug  7 01:49:34 CEST 2018 on sn-devel-144

- - - - -
5b54ced3 by Volker Lendecke at 2018-08-08T18:22:05Z
smbd: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
60c0a2b3 by Volker Lendecke at 2018-08-08T18:22:05Z
smbd: Fix CID 1438246 Unchecked return value

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
01d9be27 by Volker Lendecke at 2018-08-08T18:22:05Z
smbd: Fix CID 1438245 Dereference before null check

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ee3dd90a by Volker Lendecke at 2018-08-08T18:22:05Z
libsmb: Fix CID 1438244 Unsigned compared against 0

ndr_size_dom_sid returns a size_t, so that can't be <0. Also, the only
case that ndr_size_dom_sid returns 0 is a NULL sid
pointer. ndr_size_dom_sid can reasonably be assumed to not overflow, the
number of sub-auths is a uint8. That times 4 plus 8 always fits into a
size_t.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0c8174cf by Volker Lendecke at 2018-08-08T21:10:22Z
libsmb: Fix CID 1438243 Unchecked return value

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Aug  8 23:10:22 CEST 2018 on sn-devel-144

- - - - -
bf9cb64d by Anoop C S at 2018-08-08T23:16:22Z
s3/locking: Corrections and improvements to inline comments

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Jim McDonough <jmcd at samba.org>

- - - - -
9b105651 by Justin Stephenson at 2018-08-09T02:06:17Z
Shorten description in vfs_linux_xfs_sgid manual

this fixes a lexgrog parse error, the NAME subheader description
of the vfs_linux_xfs_sgid(8) manual was too long, this will shorten
the description and allow it to be correctly detected by mandb.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13562

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Aug  9 04:06:17 CEST 2018 on sn-devel-144

- - - - -
bd64af6b by David Disseldorp at 2018-08-09T11:29:15Z
ctdb/build: link ctdb_mutex_ceph_rados_helper against ceph-common

ceph-common linkage is needed with new versions of Ceph.
Also respect the --libcephfs_dir=<path> parameter when provided.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
85706bd2 by Samuel Cabrero at 2018-08-09T11:29:15Z
ctdb_mutex_ceph_rados_helper: Set SIGINT signal handler

Set a handler for SIGINT to release the lock.

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8d30fd59 by David Disseldorp at 2018-08-09T11:29:15Z
ctdb_mutex_ceph_rados_helper: use talloc destructor for cleanup

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
91a89c14 by David Disseldorp at 2018-08-09T11:29:15Z
ctdb_mutex_ceph_rados_helper: rename timer_ev to ppid_timer_ev

In preparation for adding a lock refresh timer.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
ce289e89 by David Disseldorp at 2018-08-09T11:29:15Z
ctdb_mutex_ceph_rados_helper: fix deadlock via lock renewals

RADOS locks without expiry persist indefinitely. This results in CTDB
deadlock during failover if the recovery master dies unexpectedly, as
subsequently elected recovery master nodes can't obtain the recovery
lock.
Avoid deadlock by using a lock expiration time (10s by default), and
renewing it periodically.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13540

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4abf348e by David Disseldorp at 2018-08-09T14:26:36Z
ctdb: add expiry test for ctdb_mutex_ceph_rados_helper

Kill the ctdb_mutex_ceph_rados_helper with SIGKILL and then confirm
that the lock is automatically released following expiry.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): David Disseldorp <ddiss at samba.org>
Autobuild-Date(master): Thu Aug  9 16:26:36 CEST 2018 on sn-devel-144

- - - - -
8479401b by Andreas Schneider at 2018-08-09T17:57:02Z
lib: Add support to parse MS Catalog files

Signed-off-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Thu Aug  9 19:57:02 CEST 2018 on sn-devel-144

- - - - -
9c131254 by Noel Power at 2018-08-10T00:43:33Z
s3/smbd: Ensure quota code is only called when quota support detected

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13563

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Aug 10 02:43:33 CEST 2018 on sn-devel-144

- - - - -
94ffd4b7 by Amitay Isaacs at 2018-08-10T00:44:36Z
dlz-bind: Add support for BIND 9.12.x

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1a86733d by Amitay Isaacs at 2018-08-10T03:36:19Z
provision: Add support for BIND 9.12.x

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Aug 10 05:36:19 CEST 2018 on sn-devel-144

- - - - -
75287495 by Douglas Bagnall at 2018-08-10T07:27:03Z
samba-tool drs showrepl tests: improve debugging for mystery error

Under some circumstances the samba-tool command is failing with no
stdout output at all, leaving few clues in the logs.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Aug 10 09:27:03 CEST 2018 on sn-devel-144

- - - - -
6b68e3ec by Anoop C S at 2018-08-10T16:05:09Z
s3/libsmb: Explicitly set delete_on_close token for rmdir

The current implementation of `rmdir` hopes to get the directory deleted
on closing last open handle when FILE_DELETE_ON_CLOSE is set on it. But
for non-empty directories Windows doesn't error out during an open call.
Following that we internally refuse to set initial delete_on_close while
opening a non-empty directory. This prevents us from trying to delete
the directory when last open handle is closed.

Instead of relying on FILE_DELETE_ON_CLOSE during an open we explicitly
set delete_on_close token on directory handle once it is available. This
ensures that NT_STATUS_DIRECTORY_NOT_EMPTY is returned for `rmdir` on
non-empty directories while closing open directory handle.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13204

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
6a7f1174 by Anoop C S at 2018-08-10T16:05:09Z
s4/torture: Add new test for DELETE_ON_CLOSE on non-empty directories

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13204

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
bca40084 by Jeremy Allison at 2018-08-10T19:08:14Z
s3: tests: smbclient. Regression test to ensure we get NT_STATUS_DIRECTORY_NOT_EMPTY on rmdir.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13204

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Aug 10 21:08:14 CEST 2018 on sn-devel-144

- - - - -
e6689c3e by Andreas Schneider at 2018-08-10T23:49:16Z
wbinfo: Free memory when we leave wbinfo_dsgetdcname()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e4f4f5eb by Andreas Schneider at 2018-08-10T23:49:16Z
s3:passdb: Don't leak memory on error in fetch_ldap_pw()

Found by covscan.

A candidate to use tallac ...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f20150fb by Andreas Schneider at 2018-08-10T23:49:16Z
s3:utils: Do not overflow the destination buffer in net_idmap_restore()

Found by covsan.

error[invalidScanfFormatWidth]: Width 128 given in format string (no. 2)
is larger than destination buffer 'sid_string[128]', use %127s to
prevent overflowing it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b7b4fc51 by Andreas Schneider at 2018-08-10T23:49:16Z
s3:utils: Do not leak memory in new_user()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d4fb124a by Andreas Schneider at 2018-08-10T23:49:16Z
s4:lib: Fix a possible fd leak in gp_get_file()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3d32c026 by Andreas Schneider at 2018-08-10T23:49:16Z
s3:client: Avoid a possible fd leak in do_get()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dbdbd487 by Andreas Schneider at 2018-08-10T23:49:16Z
s3:libads: Fix memory leaks in ads_krb5_chg_password()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3e6ce5c6 by Andreas Schneider at 2018-08-11T02:43:15Z
s3:registry: Fix possible memory leak in _reg_perfcount_multi_sz_from_tdb()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Aug 11 04:43:15 CEST 2018 on sn-devel-144

- - - - -
4c0b49b3 by Andreas Schneider at 2018-08-13T17:46:08Z
s3:winbind: Fix memory leak in nss_init()

Found by covscan.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Pair-Programmed-With: Justin Stephenson <jstephen at redhat.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4909b966 by Ralph Wuerthner at 2018-08-13T17:46:08Z
s3: vfs: time_audit: fix handling of token_blob in smb_time_audit_offload_read_recv()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13568

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a98f09a0 by Christof Schmitt at 2018-08-13T20:35:20Z
selftest: Load time_audit and full_audit modules for all tests

Previously the only test was to load these modules to trigger the
smb_vfs_assert_all_fns check. As these modules just pass through the
calls, they can be loaded for all tests to ensure that the codepaths are
exercised. This would have found the problem in
smb_time_audit_offload_read_recv.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13568

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Mon Aug 13 22:35:20 CEST 2018 on sn-devel-144

- - - - -
6b3cc791 by Volker Lendecke at 2018-08-14T06:54:17Z
g_lock: Avoid a double call to serverid_exist

If we try to G_LOCK_READ while a G_LOCK_WRITE is active, we do the
serverid_exists call twice. Avoid that.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
00513daf by Volker Lendecke at 2018-08-14T09:42:10Z
g_lock: Simplify g_lock_trylock

While chasing a bug in g_lock (not in master) I saw some opportunity to
simplify g_lock_trylock a bit. This is array handling, and array
handling is just extremely error-prone. This *might* be a little less
efficient or large numbers of READ locks, but this remains to be
seen. For now, simplify the code.

First, we make two passes now: One to remove ourselves, and the other
one to search for conflicts. Mixing up both made it pretty hard for me
to follow the code.

Second, I've removed the _mylock and mylock pointer/struct logic and
replaced it with the "mylock.pid.pid != 0 ? &mylock : NULL" when calling
g_lock_store. To me, this focuses the logic whether to add ourselves in
one place instead of spreading it around in the whole routine.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Aug 14 11:42:10 CEST 2018 on sn-devel-144

- - - - -
5edcaece by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1139 libcli/auth: Add initial tests for ntlm_password_check()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
1d89fe91 by Günther Deschner at 2018-08-14T11:57:15Z
CVE-2018-1139 libcli/auth: fix debug messages in hash_password_check()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

CVE-2018-1139: Weak authentication protocol allowed.

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
947cf385 by Günther Deschner at 2018-08-14T11:57:15Z
CVE-2018-1139 s3-utils: use enum ntlm_auth_level in ntlm_password_check().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

CVE-2018-1139: Weak authentication protocol allowed.

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c88f7795 by Günther Deschner at 2018-08-14T11:57:15Z
CVE-2018-1139 selftest: verify whether ntlmv1 can be used via SMB1 when it is disabled.

Right now, this test will succeed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

CVE-2018-1139: Weak authentication protocol allowed.

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c25460ee by Günther Deschner at 2018-08-14T11:57:15Z
CVE-2018-1139 libcli/auth: Do not allow ntlmv1 over SMB1 when it is disabled via "ntlm auth".

This fixes a regression that came in via 00db3aba6cf9ebaafdf39ee2f9c7ba5ec2281ea0.

Found by Vivek Das <vdas at redhat.com> (Red Hat QE).

In order to demonstrate simply run:

smbclient //server/share -U user%password -mNT1 -c quit \
--option="client ntlmv2 auth"=no \
--option="client use spnego"=no

against a server that uses "ntlm auth = ntlmv2-only" (our default
setting).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13360

CVE-2018-1139: Weak authentication protocol allowed.

Guenther

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
0998f2f1 by Andrej Gessel at 2018-08-14T11:57:15Z
CVE-2018-1140 Add NULL check for ldb_dn_get_casefold() in ltdb_index_dn_attr()

Signed-off-by: Andrej Gessel <Andrej.Gessel at janztec.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
3f95957d by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1140 ldb: Check for ldb_dn_get_casefold() failure in ldb_sqlite

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
b27d9733 by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1140 ldb_tdb: Ensure the dn in distinguishedName= is valid before use

ldb_dn_from_ldb_val() does not validate this untrusted input, so a later
call to ldb_dn_get_casefold() can fail if the input is not valid.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
3c1fbb18 by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1140 ldb_tdb: Check for DN validity in add, rename and search

This ensures we fail with a good error code before an eventual ldb_dn_get_casefold() which
would otherwise fail.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
b6b72d00 by Andrew Bartlett at 2018-08-14T11:57:15Z
CVE-2018-1140 ldb: Add tests for search add and rename with a bad dn= DN

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374

- - - - -
b7f0ee93 by Andrew Bartlett at 2018-08-14T11:57:15Z
Release LDB 1.5.0 for CVE-2018-1140

* Security fix for CVE-2018-1140 (NULL pointer de-reference, bug 13374)
* Fix memory leaks and missing error checks (bug 13459, 13471, 13475)

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
be4c0938 by Kai Blin at 2018-08-14T11:57:15Z
CVE-2018-1140 dns: Add a test to trigger the LDB casefolding issue on invalid chars

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466

Signed-off-by: Kai Blin <kai at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
21d628e0 by Tim Beale at 2018-08-14T11:57:15Z
CVE-2018-10919 security: Move object-specific access checks into separate function

Object-specific access checks refer to a specific section of the
MS-ADTS, and the code closely matches the spec. We need to extend this
logic to properly handle the Control-Access Right (CR), so it makes
sense to split the logic out into its own function.

This patch just moves the code, and should not alter the logic (apart
from ading in the boolean grant_access return variable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c107e2d6 by Tim Beale at 2018-08-14T11:57:15Z
CVE-2018-10919 security: Add more comments to the object-specific access checks

Reading the spec and then reading the code makes sense, but we could
comment the code more so it makes sense on its own.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
375f48f7 by Tim Beale at 2018-08-14T11:57:15Z
CVE-2018-10919 tests: Add tests for guessing confidential attributes

Adds tests that assert that a confidential attribute cannot be guessed
by an unprivileged user through wildcard DB searches.

The tests basically consist of a set of DB searches/assertions that
get run for:
- basic searches against a confidential attribute
- confidential attributes that get overridden by giving access to the
  user via an ACE (run against a variety of ACEs)
- protecting a non-confidential attribute via an ACL that denies read-
  access (run against a variety of ACEs)
- querying confidential attributes via the dirsync controls

These tests all pass when run against a Windows Dc and all fail against
a Samba DC.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
9eb8340e by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 tests: Add test case for object visibility with limited rights

Currently Samba is a bit disclosive with LDB_OP_PRESENT (i.e.
attribute=*) searches compared to Windows.

All the acl.py tests are based on objectClass=* searches, where Windows
will happily tell a user about objects they have List Contents rights,
but not Read Property rights for. However, if you change the attribute
being searched for, suddenly the objects are no longer visible on
Windows (whereas they are on Samba).

This is a problem, because Samba can tell you about which objects have
confidential attributes, which in itself could be disclosive.

This patch adds a acl.py test-case that highlights this behaviour. The
test passes against Windows but fails against Samba.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
563e454e by Gary Lockyer at 2018-08-14T11:57:16Z
CVE-2018-10919 tests: test ldap searches for non-existent attributes.

It is perfectly legal to search LDAP for an attribute that is not part
of the schema.  That part of the query should simply not match.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ba46578f by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 security: Fix checking of object-specific CONTROL_ACCESS rights

An 'Object Access Allowed' ACE that assigned 'Control Access' (CR)
rights to a specific attribute would not actually grant access.

What was happening was the remaining_access mask for the object_tree
nodes would be Read Property (RP) + Control Access (CR). The ACE mapped
to the schemaIDGUID for a given attribute, which would end up being a
child node in the tree. So the CR bit was cleared for a child node, but
not the rest of the tree. We would then check the user had the RP access
right, which it did. However, the RP right was cleared for another node
in the tree, which still had the CR bit set in its remaining_access
bitmap, so Samba would not grant access.

Generally, the remaining_access only ever has one bit set, which means
this isn't a problem normally. However, in the Control Access case there
are 2 separate bits being checked, i.e. RP + CR.

One option to fix this problem would be to clear the remaining_access
for the tree instead of just the node. However, the Windows spec is
actually pretty clear on this: if the ACE has a CR right present, then
you can stop any further access checks.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
80c4e17f by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 acl_read: Split access_mask logic out into helper function

So we can re-use the same logic laster for checking the search-ops.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4234579a by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 acl_read: Small refactor to aclread_callback()

Flip the dirsync check (to avoid a double negative), and use a helper
boolean variable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
98c2e6a1 by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 acl_read: Flip the logic in the dirsync check

This better reflects the special case we're making for dirsync, and gets
rid of a 'if-else' clause.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
fc45da52 by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 acl_read: Fix unauthorized attribute access via searches

A user that doesn't have access to view an attribute can still guess the
attribute's value via repeated LDAP searches. This affects confidential
attributes, as well as ACLs applied to an object/attribute to deny
access.

Currently the code will hide objects if the attribute filter contains an
attribute they are not authorized to see. However, the code still
returns objects as results if confidential attribute is in the search
expression itself, but not in the attribute filter.

To fix this problem we have to check the access rights on the attributes
in the search-tree, as well as the attributes returned in the message.

Points of note:
- I've preserved the existing dirsync logic (the dirsync module code
  suppresses the result as long as the replPropertyMetaData attribute is
  removed). However, there doesn't appear to be any test that highlights
  that this functionality is required for dirsync.
- To avoid this fix breaking the acl.py tests, we need to still permit
  searches like 'objectClass=*', even though we don't have Read Property
  access rights for the objectClass attribute. The logic that Windows
  uses does not appear to be clearly documented, so I've made a best
  guess that seems to mirror Windows behaviour.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
7070aa38 by Tim Beale at 2018-08-14T11:57:16Z
CVE-2018-10919 tests: Add extra test for dirsync deleted object corner-case

The acl_read.c code contains a special case to allow dirsync to
work-around having insufficient access rights. We had a concern that
the dirsync module could leak sensitive information for deleted objects.
This patch adds a test-case to prove whether or not this is happening.

The new test case is similar to the existing dirsync test except:
- We make the confidential attribute also preserve-on-delete, so it
  hangs around for deleted objcts. Because the attributes now persist
  across test case runs, I've used a different attribute to normal.
  (Technically, the dirsync search expressions are now specific enough
  that the regular attribute could be used, but it would make things
  quite fragile if someone tried to add a new test case).
- To handle searching for deleted objects, the search expressions are
  now more complicated. Currently dirsync adds an extra-filter to the
  '!' searches to exclude deleted objects, i.e. samaccountname matches
  the test-objects AND the object is not deleted. We now extend this to
  include deleted objects with lastKnownParent equal to the test OU.
  The search expression matches either case so that we can use the same
  expression throughout the test (regardless of whether the object is
  deleted yet or not).

This test proves that the dirsync corner-case does not actually leak
sensitive information on Samba. This is due to a bug in the dirsync
code - when the buggy line is removed, this new test promptly fails.
Test also passes against Windows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13434

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
860f575f by Jeremy Allison at 2018-08-14T11:57:16Z
libsmb: Ensure smbc_urlencode() can't overwrite passed in buffer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13453

CVE-2018-10858: Insufficient input validation on client directory
		listing in libsmbclient.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
be3d4b2d by Jeremy Allison at 2018-08-14T11:57:16Z
libsmb: Harden smbc_readdir_internal() against returns from malicious servers.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13453

CVE-2018-10858: Insufficient input validation on client directory
		listing in libsmbclient.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
a45de51c by Andrew Bartlett at 2018-08-14T15:02:38Z
cracknames: Fix DoS (NULL pointer de-ref) when not servicePrincipalName is set on a user

This regression was introduced in Samba 4.7 by bug 12842 and in
master git commit eb2e77970e41c1cb62c041877565e939c78ff52d.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13552

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(master): Tue Aug 14 17:02:38 CEST 2018 on sn-devel-144

- - - - -
e4f38b06 by Timur I. Bakeyev at 2018-08-14T17:08:25Z
ldb tests: fix assertion on wrong pointer

We are allocating msg02, but check in assertion msg01, which makes no
sense here.

Signed-off-by: Timur I. Bakeyev <timur at freebsd.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9eccf6a1 by Andreas Schneider at 2018-08-14T20:02:06Z
s3:libads: Free addr before we free the context

Introduced by dbdbd4875ecac3e7334750f46f1f494b7afe6628

CID 1438395

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13567

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Aug 14 22:02:06 CEST 2018 on sn-devel-144

- - - - -
b9e2a2de by Joe Guo at 2018-08-15T05:08:23Z
ldb: no need to call del_transaction in ldb_transaction_commit

No matter commit succeeded or failed, transation will be delete afterwards.
So there is no need to delete it here.

Aganst Samba this causes an `LDAP error 51 LDAP_BUSY` error when the transaction
fails, say while we try to add users to groups in large amount and
the original error is lost.

In Samba, the rootdse module fails early in the del part of the
start/end/del pattern, and in ldb_tdb and ldb_mdb a failed commit
always ends the transaction, even on failure.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d645546a by Andrej Gessel at 2018-08-15T05:08:24Z
fix mem leak in ltdb_index_dn_base_dn and ltdb_search_indexed

Signed-off-by: Andrej Gessel <Andrej.Gessel at janztec.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d71c655e by Andrej Gessel at 2018-08-15T05:08:24Z
fix mem leak in ldbsearch

Signed-off-by: Andrej Gessel <Andrej.Gessel at janztec.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
542e7c17 by Andrew Bartlett at 2018-08-15T05:08:24Z
ldb_tdb: Remove pointless check of ldb_dn_is_valid()

If the DN is not valid the ltdb_search_dn1() will catch it with ldb_dn_validate() which
is the only safe way to check this.  ldb_dn_is_valid() does not actually check, but instead
returns only the result of the previous checks, if there was one.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
2dafbd32 by Andrew Bartlett at 2018-08-15T05:08:24Z
ldb: Add new function ldb_dn_add_child_val()

This is safer for untrusted input than ldb_dn_add_child_fmt()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
9d46795b by Andrew Bartlett at 2018-08-15T05:08:24Z
ldb: extend API tests

These additional API tests just check that an invalid base DN
is never accepted.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
bdbb9422 by Andrew Bartlett at 2018-08-15T05:08:24Z
ldb: Release LDB 1.5.1

* New API ldb_dn_add_child_val() avoids passing untrusted input to
  ldb_dn_add_child_fmt() (bug 13466)
* Free memory nearer to the allocation in calls made by ldbsearch
* Do not overwrite ldb_transaction_commit failure error messages
  with a pointless del_transaction()

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
aa01203f by Andrew Bartlett at 2018-08-15T05:08:24Z
dns_server: Be strict when constructing a LDB DN from an untrusted DNS name

This changes our DNS server to be much more careful when constructing DNS names
into LDB DN values.

This avoids a segfault deep in the LDB code if the ldb_dn_get_casefold() fails there.

A seperate patch will address that part of the issue, and a later patch
will re-work this code to use single API: ldb_dn_add_child_val().  This
is not squahed with this work because this patch does not rely on a new
LDB release, and so may be helpful for a backport.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
28e2a518 by Andrew Bartlett at 2018-08-15T05:08:24Z
dns_server: Avoid ldb_dn_add_child_fmt() on untrusted input

By using the new ldb_dn_add_child_val() we ensure that the user-controlled values are
not parsed as DN seperators.

Additionally, the casefold DN is obtained before the search to trigger
a full parse of the DN before being handled to the LDB search.

This is not normally required but is done here due to the nature
of the untrusted input.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
badd7a23 by Joe Guo at 2018-08-15T05:08:25Z
samba-tool/drs: set dns_backend to SAMBA_INTERNAL in cmd_drs_clone_dc_database

The default value is "NONE", need to specify it to use SAMBA_INTERNAL so
that the DNS partitions are replicated.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
8084f183 by Joe Guo at 2018-08-15T05:08:25Z
emulate/traffic: fix next usage

In commit b0c9de820c07d77c03b80505cb811ac1dac0808f, line 343:

    self.next_conversation_id = itertools.count().next

was changed to:

    self.next_conversation_id = next(itertools.count())

which is not correct, the first one is a function, the second one is a
int. This patch fixed it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13573

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
68c64c63 by Joe Guo at 2018-08-15T05:08:25Z
traffic: uniform stats output

The original code is trying to output different data format for tty or file.
This is unnecessary and cause confusion while writing script to parse result.

The human-readable one is also easy for code to parse.
Remove if check for isatty(), just make output the same.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
ceed07fe by Joe Guo at 2018-08-15T05:08:25Z
traffic-replay: add extra check

Make sure --average-groups-per-user is not more than --number-of-users

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
371c5c70 by Joe Guo at 2018-08-15T05:08:25Z
emulate/traffic: add sAMAccountName in create_group

While using script/traffic_replay to generate users and groups, we get
autogenerated group name like:

    $2A6F42B2-39FAF4556E2BE379

This patch specify sAMAccountName to overwriten the name.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1da4ff2e by Timur I. Bakeyev at 2018-08-15T05:08:25Z
third_party:build: Test for the flags, recognized by Clang.

Make amd64 SYSTEM_UNAME_MACHINE an alias for x86_64.

Signed-off-by: Timur I. Bakeyev <timur at iXsystems.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
14077b66 by Tim Beale at 2018-08-15T05:08:25Z
netcmd: domain backup didn't support prompting for password

The online/rename backups only worked if you specified both the username
and password in the actual command itself. If you just entered the
username (expecting to be prompted for the password later), then the
command was rejected.

The problem was the order the code was doing things in. We were checking
credopts.creds.get_password() *before* we'd called
credopts.get_credentials(lp), whereas it should be the other way
around.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13566

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
8fb706c3 by Tim Beale at 2018-08-15T05:08:25Z
netcmd: Fix kerberos option for domain backups

The previous fix still didn't work if you specified --kerberos=yes (in
which case the creds still doesn't have a password).

credopts.get_credentials(lp) should be enough to ensure a user/password
is set (it's all that the other commands seem to do).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13566

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d2d03951 by Tim Beale at 2018-08-15T05:08:26Z
netcmd: Delete unnecessary function

Minor code cleanup. The last 2 patches gutted this function, to the
point where there's no longer any value in keeping it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13566

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
f249bea1 by Tim Beale at 2018-08-15T08:19:09Z
netcmd: Fix --kerberos=yes and --no-secrets domain backups

The --kerberos=yes and --no-secrets options didn't work in combination
for domain backups. The problem was creds.get_username() might not
necessarily match the kerberos user (such as in the selftest
environment). If this was the case, then trying to reset the admin
password failed (because the creds.get_username() didn't exist in
the DB).

Because the admin user always has a fixed RID, we can work out the
administrator based on its object SID, instead of relying on the
username in the creds.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13566

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Aug 15 10:19:09 CEST 2018 on sn-devel-144

- - - - -
739691fd by Andrew Bartlett at 2018-08-16T21:42:19Z
buildtools: Split git ls-files output on newline, not any whitespace

This allows files to have a space in the filename within the Samba git tree.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
6d52ef8d by Garming Sam at 2018-08-16T21:42:19Z
git: Treat .dump files as binary

This means that git grep will no longer show TDB dumps. This can be
changed at runtime using -a for all to include these files, while -I
will also omit any references to the files (no Binary file * matches).

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ea297d0c by Garming Sam at 2018-08-16T21:42:19Z
winreg: Add hyper REG_QWORD to parsing routines

This will be useful when exporting registry.pol files.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
572fd631 by Garming Sam at 2018-08-16T21:42:19Z
preg: Build python preg bindings

These will be used in the GPO import/export.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
463dcc90 by Garming Sam at 2018-08-16T21:42:19Z
preg: Unpack winreg_Data for parsing

It seems that there might be pre-existing endianness issues which would be fixed by the ndr_push.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c71ba94c by Garming Sam at 2018-08-16T21:42:20Z
preg: Using winreg_Data_GPO instead of DATA_BLOB

We need to make a duplicate in order to have reasonable python bindings.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8140a7bb by Garming Sam at 2018-08-16T21:42:20Z
preg: Use gensize to allow modification of winreg data to be repacked

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7047f457 by Garming Sam at 2018-08-16T21:42:20Z
gp_parse: Introduce new module for parsing GPO files

This is the default parser which will cause the file to be restored
as-is -- leaving only an effectively blank XML file as a placeholder.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
34453a08 by Garming Sam at 2018-08-16T21:42:20Z
gpo: Add a backup command (similar to fetch)

The idea behind this command is that you will eventually backup a number
of XML files which can be user-editable and have generic entities to be
later restored in the same domain or a different domain.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
aac6cd37 by Garming Sam at 2018-08-16T21:42:21Z
gpo: Add a restore command (for backups) from XML

Currently because no parsers have been written, this just copies the old
files and puts them in their places.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6c5a5077 by Garming Sam at 2018-08-16T21:42:21Z
gp_pol: Parse the .pol files (PReg) which stored winreg settings

Currently, we do not look inside the .pol files for any settings (and do
not generalize any so far).

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e27c8689 by Garming Sam at 2018-08-16T21:42:21Z
gp_ini: Parse .ini files in SYSVOL

These are fdeploy, scripts + psscripts as well as the GPT.ini at the top
level. Note that GPT.ini has a different character encoding and we
specify it here.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4a69308b by Garming Sam at 2018-08-16T21:42:21Z
gp_csv: Parse the audit.csv file which records audit settings

Based on the setting, the csv will omit certain fields. Using this we
can later infer as to how to generalize the ACLs and SIDs.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dac3c204 by Garming Sam at 2018-08-16T21:42:21Z
gp_inf: Parse the GptTmpl.inf file which stores security settings

This is NOT an ini file and CANNOT be parsed by Python ConfigParser
without losing information (it would likely eat meaningful whitespace
and so should not be done).

There are three main types of settings:

 * Name,Mode,ACL
 * key = value
 * registry key and value

   Note: This appears as key=value, but registry keys in the general
   case may have = in their names, so we record the entire string in
   order to be as safe as possible.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
497db985 by Garming Sam at 2018-08-16T21:42:21Z
gp_aas: Leave a placeholder for the .aas files for now

This is to be implemented, but the documentation is somewhat lacking for
the .aas files and we so we leave this for now. In particular, the
documentation doesn't seem to describe all the possible sections, nor do
we understand what happens if we replace certain aspects of the file --
and whether or not it will remain functional.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
854c3eb2 by Garming Sam at 2018-08-16T21:42:21Z
gpo: Enable more specific parsers of GPO files

* .pol files
* .ini (and GPT.ini)
* audit.csv
* GptTmpl.inf

.aas is currently not handled.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ee010392 by Garming Sam at 2018-08-16T21:42:21Z
gp_csv: Add CSV generalization metadata

There are user identifiers and ACLs which may be stored in the audit CSV.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4edb1769 by Garming Sam at 2018-08-16T21:42:21Z
gp_ini: Allow better overriding of behaviour in inherited classes

We will need this to parse the parameters or section names as SIDs for fdeploy1.ini

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
806e3e60 by Garming Sam at 2018-08-16T21:42:21Z
gp_ini: Add a fdeploy1 parser for better generalization

We still fail to handle entities in fdeploy.ini (version 0) files. Here we
manage to factor out some of the SIDs, but not all of them. This will be
completed in a later patch. The overall idea is to split the SID values into
individual XML elements and annotate them. We also note down network paths for
the redirection folders.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
57dd88ce by Garming Sam at 2018-08-16T21:42:22Z
gp_ini: Add a scripts ini parser for better generalization

We mark the command path argument as a network path.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
abff0c4f by Garming Sam at 2018-08-16T21:42:22Z
gp_parse: Add a generalize XML function to the top level parser

In this function we take XML and using the required metadata, we rewrite
it into a generic form using entities. ElementTree unfortunately does
not allow us to store unescaped entities, and so we must do a textual
replace on the output XML.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3ff695fd by Garming Sam at 2018-08-16T21:42:22Z
fdeploy_ini: Generalize the share name SIDs

This overrides the custom entity handler defined in the top level parser.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
07de1565 by Garming Sam at 2018-08-16T21:42:22Z
gpo: Add a --generalize to the backup command

This normally prints out the entities in DTD form to be given to the restore
command with --entities. Specifying --entities during the backup conveniently
writes these entities to a file. Generalizing occurs after the standard backup
on the XML files, which will then re-write the XML file.

There are a number of files which can be further handled, including many of the
preferences XML files. This will require more annotation and parsing.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9977b4bc by Garming Sam at 2018-08-16T21:42:22Z
gitattributes: Ignore .SAMBABACKUP files

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cedfea1b by Garming Sam at 2018-08-16T21:42:22Z
gpo: Make restore with entities more robust

Sometimes the restore fails for unknown reasons, but rearranging the XML
such that the DTD is after the xml header appears to fix it. This might
be the case in certain files where no entities are used perhaps.

This could probably be made more tolerant using regex, but for the most
part we expect the fixed output from the minidom pretty-printed XML.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a844137e by Garming Sam at 2018-08-16T21:42:22Z
tests/gpo: Add a backup for showing that GPO backup and restore works

This will be used to write a test in the coming patches.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
53ee5157 by Garming Sam at 2018-08-16T21:42:22Z
tests/gpo: Tests using a static backup directory from gpo backup

In particular, we want to see that the binary matches, that the XML will
backup to the same values, that the fallback with copy-restore works,
and that the generalize will generalize over different restored
entities.

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f4f0a5d9 by Garming Sam at 2018-08-16T21:42:22Z
gpo: Maintain an XML DTD for reference of the backup

This may or may not actually parse, but is mostly for reference

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cdff99b9 by Andrew Bartlett at 2018-08-16T21:42:22Z
gpo: Always use an SMB signed connection

This ensures data integrity in the backup.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
3990df08 by Andrew Bartlett at 2018-08-17T00:44:41Z
WHATSNEW: Add information on new GPO features

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Aug 17 02:44:41 CEST 2018 on sn-devel-144

- - - - -
d313e0e4 by Joe Guo at 2018-08-17T00:58:26Z
descriptor: add missing backslash for long sddl str

Find this bug while doing PEP8.
We are lucky this code was not used yet.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
2f371493 by Joe Guo at 2018-08-17T00:58:26Z
python/samba/tests: fix SamDB dummy replacement

In commit 6de9d878b, a dummy SamDB lambda was added:

    SamDB = lambda *x: None

The `*x` will only cover positional args. If we call it with kwargs:

    samdb = SamDB(url=url)

We will get TypeError:

    <lambda>() got an unexpected keyword argument 'url'

This commit fix this. It also fix PEP8 E731:

    do not assign a lambda expression, use a def

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13542

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
cd3b06fa by Joe Guo at 2018-08-17T00:58:26Z
python3: reuse cmp_fn defined in compat.py

This will also fix PEP8 E306:

    expected 1 blank line before a nested definition, found 0

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a39c8f44 by Tim Beale at 2018-08-17T00:58:26Z
Fix PEP8 warning E711 comparison to None

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
96b726ea by Tim Beale at 2018-08-17T00:58:26Z
Fix PEP8 warning E201/202/203 array/dict whitespace

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
9b86c5f8 by Tim Beale at 2018-08-17T00:58:26Z
Fix PEP8 warning E122/E126/E127 wrong indent for continuation lines

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
078bd795 by Tim Beale at 2018-08-17T00:58:26Z
python/pso tests: use  string .format() style rather than C-style %s/%d.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
3a050542 by Andrew Bartlett at 2018-08-17T00:58:27Z
samba-tool domain passwordsettings: Avoid except Exception

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
30e6e04c by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning F841 local variable 'blah' is assigned to but never used

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1a30a68b by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning E225 missing whitespace around operator

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e741a193 by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning F401 'blah' imported but unused

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
092130be by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning E231 missing whitespace after ','

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
b8920aab by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning E302 expected 2 blank lines

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
87b919b6 by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning E303 too many blank lines

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
26ece82e by Tim Beale at 2018-08-17T00:58:27Z
Fix PEP8 warning W291 trailing whitespace

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
2bd0a208 by Tim Beale at 2018-08-17T00:58:28Z
Fix PEP8 warning W503 line break before binary operator

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a86e2b34 by Tim Beale at 2018-08-17T00:58:28Z
Refactor for PEP8 warning E501 line too long

The attribute names here are so long it means there's not a very nice
way to wrap the long lines, so add a helper function

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7065f529 by Tim Beale at 2018-08-17T00:58:28Z
Refactor for PEP8 warning E501 line too long

Rename a parameter that an internal function takes so that the function
call doesn't overrun 80 chars.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
68f8a1c2 by Tim Beale at 2018-08-17T00:58:28Z
Fix PEP8 warning E501 line too long

Mostly involves splitting up long strings or comments so that they
span multiple lines. Some place-holder variables have been added in a
few places to avoid exceeding 80 chars.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
60b4a1be by Tim Beale at 2018-08-17T00:58:28Z
Refactor for PEP8 warning E501 line too long

Add a wrapper function to avoid long lines. This also helps
a little to manage/contain the complexity of the code.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
f9ff50ae by Tim Beale at 2018-08-17T00:58:28Z
Refactor for PEP8 warning E501 line too long

Rename a couple of really long functions.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
b0130fe4 by Andrew Bartlett at 2018-08-17T03:53:54Z
docs smb.conf: Clarify that wreplsrv:periodic_interval is in seconds

As requested by oota on samba-technical

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Aug 17 05:53:54 CEST 2018 on sn-devel-144

- - - - -
fc41281e by Volker Lendecke at 2018-08-17T05:39:16Z
torture3: Simplify the g_lock6 test

Do string_term_tdb_data just once, this is a leftover from a sweeping
change from "char *" to TDB_DATA as g_lock key.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13195
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8734970c by Volker Lendecke at 2018-08-17T05:39:17Z
torture3: Enhance g_lock6 to run without CLEAR_IF_FIRST

CLEAR_IF_FIRST doesn't really work in the cluster. This needs to be
applied to all tests, but lock6 is what I care about right now.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13195
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
682fafe2 by Volker Lendecke at 2018-08-17T08:34:53Z
torture3: Extend the g_lock6 test to also cover upgrades

The fixes for #13195 were incomplete and did not cover upgrades
properly. It's all gone in master with the new code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13195
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Aug 17 10:34:53 CEST 2018 on sn-devel-144

- - - - -
0f6b4b43 by Volker Lendecke at 2018-08-17T09:30:10Z
pygpo: Fix a talloc_tos() leak in py_gpo_get_unix_path

cache_path() implicitly puts its result on talloc_tos(). As in
py_gpo_get_unix_path the talloc_stackframe() is only created after the
cache_path() call, we leak the result of cache_path() on
talloc_tos() (which might or might not exist).

This converts the function to the pattern used elsewhere: Create the
stackframe as the very first action and remove it as the very last
action in the function.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
0a9273f8 by Volker Lendecke at 2018-08-17T09:30:10Z
lib: Pass "mem_ctx" down to xx_path

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
f986a73b by Volker Lendecke at 2018-08-17T09:30:10Z
lib: Pass mem_ctx to lock_path()

Fix a confusing API: Many places TALLOC_FREE the path where it's not
clear you have to do it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
c2ea1007 by Volker Lendecke at 2018-08-17T09:30:11Z
lib: Pass mem_ctx to state_path()

Fix a confusing API: Many places TALLOC_FREE the path where it's not
clear you have to do it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
6ca5ba52 by Volker Lendecke at 2018-08-17T12:28:51Z
lib: Pass mem_ctx to cache_path()

Fix a confusing API: Many places TALLOC_FREE the path where it's not
clear you have to do it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Aug 17 14:28:51 CEST 2018 on sn-devel-144

- - - - -
29eb2fc6 by Volker Lendecke at 2018-08-17T16:25:08Z
smbd: Enhance debugging in set_file_size

I've stumbled over a case where VFS_FTRUNCATE wasn't called due to an
unchanged size. Make that easier to detect. Also, get rid of an ancient
cast to (double).

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
04f62984 by Volker Lendecke at 2018-08-17T16:25:08Z
smbd: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a56bd0a9 by Volker Lendecke at 2018-08-17T16:25:08Z
smbd: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fea8ebce by Volker Lendecke at 2018-08-17T16:25:08Z
g_lock: Fix DEBUG messages

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0d38e9e8 by Volker Lendecke at 2018-08-17T16:25:08Z
dbwrap: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Rowland Penny <rpenny at samba.org>

- - - - -
22be863b by Volker Lendecke at 2018-08-17T16:25:08Z
smbd: Fix a few DEBUG statements

%u is not necessarily correct for uint32_t, avoid casts

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
14f9e6f8 by Volker Lendecke at 2018-08-17T19:29:15Z
dbwrap: Clarify db_open_watched API

Point out in the API that "backend" talloc_moves into the watched
database.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Aug 17 21:29:15 CEST 2018 on sn-devel-144

- - - - -
41aa55f4 by Paulo Alcantara at 2018-08-17T23:32:25Z
s3: util: Do not take over stderr when there is no log file

In case we don't have either a /var/log/samba directory, or pass a
non-existent log directory through '-l' option, all commands that are
daemonized with '-D' option hang when executed within a subshell.

An example on how to trigger that:

  # rm -r /var/log/samba
  # s=$(nmbd -D -s /etc/samba/smb.conf -l /foo123)
  (never returns)

So, when the above command is executed within a subshell the following
happens:

  (a) Parent shell creates a pipe, sets write side of it to fd 1
    (stdout), call read() on read-side fd, forks off a new child process
    and then executes nmbd in it.
  (b) nmbd sets up initial logging to go through fd 1 (stdout) by
    calling setup_logging(..., DEBUG_DEFAULT_STDOUT). 'state.fd' is now
    set to 1.
  (c) reopen_logs() is called by the first time which then calls
    reopen_logs_internal()
  (d) in reopen_logs_internal(), it attempts to create log.nmbd file in
    /foo123 directory and fails because directory doesn't exist.
  (e) Regardless whether the log file was created or not, it calls
    dup2(state.fd, 2) which dups fd 1 into fd 2.
  (f) At some point, fd 0 and 1 are closed and set to /dev/null

The problem with that is because parent shell in (a) is still blocked in
read() call and the new write side of the pipe is now fd 2 -- after
dup2() in (e) -- and remains unclosed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13578

Signed-off-by: Paulo Alcantara <palcantara at suse.de>
Reviewed-by: Jim McDonough <jmcd at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Aug 18 01:32:25 CEST 2018 on sn-devel-144

- - - - -
8262efbc by Swen Schillig at 2018-08-18T00:01:26Z
ctdb: Replace calculation of bytes to read from socket by MIN() macro

The calculation of the bytes to read from the socket can be done easier
by the usage of the common MIN() macro.

Signed-off-by: Swen Schillig <swen at vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d0ed4a53 by Swen Schillig at 2018-08-18T02:58:05Z
ctdb: calculate queue input buffer size correctly

The queue's input buffer is calculated in an iterative way.
This can result in a few back and forth jumping and
a few memory allocations and mem-free cycles.
This is very time consuming and not required, because the required
memory size can be calculated right away.

Signed-off-by: Swen Schillig <swen at vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Aug 18 04:58:05 CEST 2018 on sn-devel-144

- - - - -
9ee4d946 by Andreas Schneider at 2018-08-18T13:21:39Z
python: Fix print in dns_invalid.py

https://bugzilla.samba.org/show_bug.cgi?id=13580

Signed-off-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Sat Aug 18 15:21:39 CEST 2018 on sn-devel-144

- - - - -
51d57073 by Volker Lendecke at 2018-08-20T21:28:25Z
vfs_fruit: Fix a leak of "br_lck"

Fix a panic if fruit_access_check detects a locking conflict.

do_lock() returns a valid br_lck even in case of a locking conflict.
Not free'ing it leads to a invalid lock order panic later, because
"br_lck" corresponds to a dbwrap lock on brlock.tdb.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13584

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ec3c37ee by Volker Lendecke at 2018-08-21T00:33:04Z
torture: Demonstrate the invalid lock order panic

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13584

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Aug 21 02:33:05 CEST 2018 on sn-devel-144

- - - - -
7460d9b9 by Volker Lendecke at 2018-08-21T22:57:31Z
smbd: Remove koplocks->contend_level2 callbacks

This was only implemented by onefs in this way. If we get around to use
for example fanotify or something similar, we can either re-add them or
do it in a different way. For now, simplify the code.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Wed Aug 22 00:57:31 CEST 2018 on sn-devel-144

- - - - -
f16d9172 by Volker Lendecke at 2018-08-21T22:58:41Z
libgpo: Fix CID 1438462 Error handling issues (CHECKED_RETURN)

Yes, this creates a leak of "data", but the other error exits in this
function are the same.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
09acfb76 by Volker Lendecke at 2018-08-21T22:58:41Z
dsdb: Fix CID 1438461 Error handling issues (CHECKED_RETURN)

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
781faca0 by Volker Lendecke at 2018-08-21T22:58:41Z
dsdb: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
75ced0d1 by Volker Lendecke at 2018-08-21T22:58:41Z
libads: Fix an error path talloc leak

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fb81fb2d by Volker Lendecke at 2018-08-22T01:59:51Z
libads: Simplify parse_spn()

A few lines less and quite some bytes less .text

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Aug 22 03:59:51 CEST 2018 on sn-devel-144

- - - - -
9c71f61e by Jeremy Allison at 2018-08-22T19:50:41Z
s3: smbd: Ensure get_real_filename() copes with empty pathnames.

Needed for vfs_glusterfs, as Gluster requires "." not '\0'.

Based on a fix from Anoop C S <anoopcs at redhat.com>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13585

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ira Cooper <ira at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Aug 22 21:50:41 CEST 2018 on sn-devel-144

- - - - -
c39ec642 by Volker Lendecke at 2018-08-23T10:08:21Z
torture: Make sure that fruit_ftruncate only unlinks streams

Follow-up to

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13441

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
8c142348 by Volker Lendecke at 2018-08-23T13:28:47Z
vfs_fruit: Don't unlink the main file

The original fix for bug 13441 was missing a check that verifies that
fruit_ftruncate() is actually called on a stream.

Follow-up to

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13441

Pair-Programmed-With: Volker Lendecke <vl at samba.org>

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Thu Aug 23 15:28:48 CEST 2018 on sn-devel-144

- - - - -
831bb357 by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E271: multiple spaces after keyword

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e551f5f5 by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E713: test for membership should be 'not in'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5027405d by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E714: test for object identity should be 'is not'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d7863ffe by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E731: do not assign a lambda expression, use a def

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
3606a49c by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix W601: .has_key() is deprecated, use 'in'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1c5fa5dc by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix W602: deprecated form of raising exception

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
086daf5f by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E101: indentation contains mixed spaces and tabs

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
ada5af53 by Joe Guo at 2018-08-24T05:49:25Z
PEP8: whitespace fixes in wintest.py

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
4fc08d8f by Joe Guo at 2018-08-24T05:49:25Z
PEP8: fix E111: indentation is not a multiple of four

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
eba81f7f by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E115: expected an indented block (comment)

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5037731e by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E116: unexpected indentation (comment)

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
b43408b3 by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E121: continuation line under-indented for hanging indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
297faf32 by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E122: continuation line missing indentation or outdented

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
484ce063 by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E123: closing bracket does not match indentation of opening bracket's line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
dba0c7eb by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E124: closing bracket does not match visual indentation

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
aa163f1a by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E125: continuation line with same indent as next logical line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
bbb9f576 by Joe Guo at 2018-08-24T05:49:26Z
PEP8: fix E127: continuation line over-indented for visual indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5d532543 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E128: continuation line under-indented for visual indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
944d7043 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E131: continuation line unaligned for hanging indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a9551eda by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E201: whitespace after '('

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
562411bd by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E202: whitespace before ')'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
ba0827b5 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E203: whitespace before ':'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d9c282a9 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E211: whitespace before '('

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fb5ea356 by Joe Guo at 2018-08-24T05:49:27Z
PEP8: fix E222: multiple spaces after operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
32266d2d by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E225: missing whitespace around operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
87bbc2df by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E226: missing whitespace around arithmetic operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e1edeae8 by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E227: missing whitespace around bitwise or shift operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
35de4227 by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E228: missing whitespace around modulo operator

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
12d3fbe1 by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E231: missing whitespace after ','

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
40ef91fb by Joe Guo at 2018-08-24T05:49:28Z
PEP8: fix E241: multiple spaces after ','

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1ccc36b4 by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E251: unexpected spaces around keyword / parameter equals

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
c809a860 by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E261: at least two spaces before inline comment

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7a07d422 by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E265: block comment should start with '# '

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
542e91ef by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E301: expected 1 blank line, found 0

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
211c9a5f by Joe Guo at 2018-08-24T05:49:29Z
PEP8: fix E302: expected 2 blank lines, found 1

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fd6b2086 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E303: too many blank lines (2)

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
115f2a71 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E305: expected 2 blank lines after class or function definition, found 1

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
4d529c40 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E306: expected 1 blank line before a nested definition, found 0

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
95c36d82 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E401: multiple imports on one line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
7b031b01 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E502: the backslash is redundant between brackets

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d23170c9 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E701: multiple statements on one line (colon)

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
cabb2997 by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E703: statement ends with a semicolon

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e940e4cd by Joe Guo at 2018-08-24T05:49:30Z
PEP8: fix E711: comparison to None should be 'if cond is not None:'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1676a4dc by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix E712: comparison to False should be 'if cond is False:' or 'if not cond:'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
9f5bbcc1 by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix E713: test for membership should be 'not in'

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
aa243d1a by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix W291: trailing whitespace

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
6f9eb617 by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix W293: blank line contains whitespace

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
c9f2fdea by Joe Guo at 2018-08-24T05:49:31Z
PEP8: fix W391: blank line at end of file

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
4284f604 by Joe Guo at 2018-08-24T05:49:31Z
PEP8: add pycodestyle config in setup.cfg

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
9c750ad7 by Douglas Bagnall at 2018-08-24T05:49:31Z
PEP8: more space before equals

This dropped out during rebase

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2ccb0b03 by Douglas Bagnall at 2018-08-24T05:49:31Z
PEP8: improve spacing around colons

These dropped out of Joe's patches during rebase and review.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5d0dc38c by Douglas Bagnall at 2018-08-24T05:49:31Z
ldb tests: remove unused code from match_rules.py

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
333bc327 by Douglas Bagnall at 2018-08-24T05:49:31Z
PEP8: better formatting of (CONST1|CONST2)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
08d1573b by Douglas Bagnall at 2018-08-24T05:49:31Z
PEP8: add spaces after operators

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d246eec0 by Douglas Bagnall at 2018-08-24T05:49:32Z
dcerpc/dns_server tests: use record malformed as probably intended

Both ways the record is malformed, but it is more likely we meant
AAAAAAAAAA... 1
than
A 1A 1A 1A 1A ...

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
944a4609 by Douglas Bagnall at 2018-08-24T05:49:32Z
dcerpc py tests: improve argument formatting

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
06320851 by Douglas Bagnall at 2018-08-24T05:49:32Z
PEP8: improve formatting around bit-wise OR ("|")

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
aa5db3da by Douglas Bagnall at 2018-08-24T05:49:32Z
PEP8: improve formatting around various operators

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0f3f63f2 by Douglas Bagnall at 2018-08-24T08:58:16Z
PEP8: line up a couple of lists

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Aug 24 10:58:16 CEST 2018 on sn-devel-144

- - - - -
48335725 by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-common: Fix aliasing issue in IPv6 checksum

Since commit 9c51b278b1700cd5f3e2addc19b7c711cc2ea10b the compiler has
been able to inline the affected call to uint16_checksum().  Given
that the data (phdr) is being accessed by an incompatible
pointer (data) there is an aliasing problem when the call is inlined.
This results in incorrect behaviour with -O2/-O3 when compiling with
at least GCC 6, 7, and 8.

Fix this by making the types compatible.

Also fixes CID 1437604 (Reliance on integer endianness).  This is a
false positive because the uint16_checksum doesn't depend on the order
of the input uint16_t items.

https://bugzilla.samba.org/show_bug.cgi?id=13588

Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d4afb60a by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-doc: Make config migration script notice removed CTDB_BASE option

This should never have been a user-level option, but some people used
it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
64d4a7ae by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-doc: Handle boolean options in config migration more carefully

Values for ctdb.conf options are now returned by
get_ctdb_conf_option().  The main goal is to allow old boolean options
to be replaced by new logically negated options.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
17068e75 by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-config: Change option "no realtime" option to "realtime scheduling"

Negative options can be confusing, so switch to a positive option.

This was supposed to be done months ago but was forgotten.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
43adcd71 by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-doc: Change option "no realtime" option to "realtime scheduling"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8ddfc26d by Martin Schwenke at 2018-08-24T08:59:20Z
ctdb-doc: Add support for migrating tunables to ctdb.conf options

This will become common, so will be useful to have support for.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f42486e8 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-config: Switch tunable TDBMutexEnabled to a config option

Use the "database:tdb mutexes" option instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a9758f41 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-doc: Switch tunable TDBMutexEnabled to a config option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
21de59ab by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-common: Allow boolean configuration values to have yes/no values

This make the new configuration style more consistent with the old one.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
914e9f22 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-daemon: Pass DisableIPFailover tunable via environment variable

Preparation for obsoleting this tunable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8e160d33 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-tests: Drop DisableIPFailover simple test

This is about to become a config file option that can't be dynamically
changed at run-time, so drop this test for now.  This test will be added
once the tunable becomes a config file option.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
893dd623 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-failover: Add failover configuration options

Only a "disabled" option for now.  Not documented because it isn't
used yet.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d003a41a by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-config: Integrate failover options into conf-tool

Update and add tests accordingly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
92963412 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-config: Switch tunable DisableIPFailover to a config option

Use the "failover:disabled" option instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
78aad762 by Martin Schwenke at 2018-08-24T08:59:21Z
ctdb-doc: Switch tunable DisableIPFailover to a config option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
55893bf8 by Martin Schwenke at 2018-08-24T08:59:22Z
ctdb-tests: Add an extra conf loading test case

This shows that config file loading continues in spite of unknown keys
if ignore_unknown is true.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
6fb80cbf by Martin Schwenke at 2018-08-24T12:13:12Z
ctdb-tests: Check that no IPs are assigned when failover is disabled

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Fri Aug 24 14:13:12 CEST 2018 on sn-devel-144

- - - - -
7c89edfe by Bernd Kuhls at 2018-08-24T15:22:09Z
Fix uClibc build on 64bit platforms by including stdint.h

Fixes an error detected by buildroot autobuilders:
http://autobuild.buildroot.net/results/573/573e2268e205e10d1352fa81122d8f225fdb4575/build-end.log

/home/rclinux/rc-buildroot-test/scripts/instance-1/output/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/stdint.h:122:27:
error: conflicting types for 'uintptr_t'
 typedef unsigned long int uintptr_t;
                           ^
In file included from ../lib/ldb/tests/ldb_msg.c:17:0:
../third_party/cmocka/cmocka.h:126:28: note: previous declaration of 'uintptr_t' was here
       typedef unsigned int uintptr_t;

The define __WORDSIZE is missing when cmocka.h decides how to
define uintptr_t, this patch includes stdint.h when needed.

Patch sent upstream:
https://lists.samba.org/archive/samba-technical/2018-January/125306.html

Signed-off-by: Bernd Kuhls <bernd.kuhls at t-online.de>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Aug 24 17:22:10 CEST 2018 on sn-devel-144

- - - - -
e5e66191 by Christof Schmitt at 2018-08-25T01:23:05Z
smbd: Remove unused KOPLOCK flags

This effectively reverts commit 17eba16b. It looks like these flags have
been introduced as part of the onefs support which has been removed
again. As there is no other use for the flags, remove them.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Sat Aug 25 03:23:05 CEST 2018 on sn-devel-144

- - - - -
a4f9e380 by Andrew Bartlett at 2018-08-25T07:00:16Z
selftest: Ensure winbindd is talking to the DC (itself) at startup

This might reduce issues with the first winbind-using test failing

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
9a19563e by Andrew Bartlett at 2018-08-25T07:00:16Z
selftest: Ensure all python3 tests are tagged (named) python3

This will help when splitting them into a distinct build.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
da6530d9 by Andrew Bartlett at 2018-08-25T07:00:16Z
selftest: Move samba.tests.dcerpc.rpcecho to chgdcpass:local

Before this patch, no python3-compatible tests ran against any of fl2000dc, vampire_2000_dc or chgdcpass
and so an autobuild of the samba-ad-dc-2-py3 environment would fail with no tests to run.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3d2baa9c by Andrew Bartlett at 2018-08-25T07:00:16Z
selftest: Move samba.tests.lsa_string to none environment, it does not contact a server

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
05389bcb by Andrew Bartlett at 2018-08-25T09:53:15Z
autobuild: Allow automatic handling of autobuild tasks ending in -py3

This will allow splitting up of the python2 and python3 tests without
duplication of this already complex file.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Aug 25 11:53:15 CEST 2018 on sn-devel-144

- - - - -
9a3b6470 by Andrew Bartlett at 2018-08-27T06:53:20Z
autobuild: Fix -py3 support to look in tasks (the table with the tests) not tasknames (the requested list)

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c833ec4c by Andrew Bartlett at 2018-08-27T10:00:11Z
travis-ci: Add python3-crypto to package list

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Aug 27 12:00:11 CEST 2018 on sn-devel-144

- - - - -
59f13347 by Jeremy Allison at 2018-08-27T15:28:24Z
s3: VFS: vfs_full_audit: Add $cwd arg to smb_fname_str_do_log().

Not yet used.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13565

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
4d72ebb8 by Jeremy Allison at 2018-08-27T18:23:55Z
s3: VFS: vfs_full_audit: Ensure smb_fname_str_do_log() only returns absolute pathnames.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13565

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Mon Aug 27 20:23:55 CEST 2018 on sn-devel-144

- - - - -
a5c4d91e by Volker Lendecke at 2018-08-27T21:09:15Z
libsmb: Add protocol-agnostic cli_read

So far only cli_pull could be called directly without looking at the
protocol. We did not have a simple read that did the right thing
depending on the protocol

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fd203725 by Volker Lendecke at 2018-08-27T21:09:15Z
libsmb: Rename cli_writeall_send/recv to cli_smb1_writeall_send/recv

Preparing a protocol agnostic writeall

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
082f60af by Volker Lendecke at 2018-08-27T21:09:15Z
libsmb: Expose protocol-agnostic cli_writeall_send/recv

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ffa1c040 by Volker Lendecke at 2018-08-28T00:03:07Z
examples: Add winexe re-implemented on current Samba libs

winexe from https://sourceforge.net/projects/winexe/ is a project
based on Samba libraries from 2012. According to the winexe git
repository the last Samba commit winexe was updated to is 47bbf9886f0c
from November 6, 2012. As winexe uses unpublished Samba internal
libraries, it broke over time.

This is a port of the winexe functionality to more modern Samba
versions. It still uses internal APIs, but it being part of the tree
means that it is much easier to keep up to date.

The Windows service files were taken literally from the original
winexe from the sourceforge git. Andrzej Hajda chose GPLv3 only and
not GPLv3+. As GPL evolves very slowly, this should not be a practical
problem for quite some time.

To build it under Linux, you need mingw binaries on your build
system. Under Debian stretch, the package names are gcc-mingw-w64 and
friends.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Aug 28 02:03:07 CEST 2018 on sn-devel-144

- - - - -
0da15fa7 by Andrew Bartlett at 2018-08-29T03:28:16Z
autobuild: Implement a split python2 and python2 build pattern

The default tasks will run the tests without --extra-python specified and
the new -py3 tasks will run the python3 tests only.

This will reduce the complexity of the build combinations

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
ce53dd9c by Andrew Bartlett at 2018-08-29T03:28:16Z
gitlab-ci: Run the new python3 autobuild tasks

These additional tasks should be less complex than the full build and help get us to
a pure python3 build eventually

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
38784f2b by Andrew Bartlett at 2018-08-29T03:28:16Z
autobuild: Reduce duplication of task list in autobuild

The defaulttasks or builddirs are often updated out of sync, which causes confusion until
it is resolved.

We simply choose "." as the builddir for the tasks that
are not in the default set.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
f9e494a6 by Andrew Bartlett at 2018-08-29T03:28:16Z
autobuild: Move ad_dc_no_nss into to samba-ad-dc-2 autobuild job

This tries to to split up the tasks more evenly and may help with the python3 tests
against this environment if started from a more isolated job.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
6b116436 by Andrew Bartlett at 2018-08-29T03:28:16Z
autobuild: Move backup/restore environments into to samba-ad-dc-2 autobuild job

This tries to to split up the tasks more evenly and may help with the python3
work by isolating them from the long samba job.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
acb8cf1e by Andrew Bartlett at 2018-08-29T03:28:17Z
travis-ci: Add py3 jobs to match new jobs in autobuild.py

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
47c14ef6 by Andrew Bartlett at 2018-08-29T03:28:17Z
autobuild: use close_fds=True to avoid *.stderr and *.stdout inheriting into every process

This closes fds other than 0, 1, 2.

This ensures only the correct *.stderr and *.stdout is attached, via
the stdout/stderr parameter to Popen(), but not every other FD
currently open in python at the time Popen is called.

For the tail invocation and other calls to Popen(), because fds 0, 1,
2 are still attached, these function as before.

Per https://docs.python.org/2.6/library/subprocess.html:

"If close_fds is true, all file descriptors except 0, 1 and
2 will be closed before the child process is executed. (Unix only)."

And regarding the passed in parameters:

"stdin, stdout and stderr specify the executed programs’ standard
input,
standard output and standard error file handles, respectively.  "
...

"With None (the default), no redirection will occur;
the child’s file handles will be inherited from the parent. "

(The unwanted inherited files would be on a random high FD, where the
program wouldn't know what to do with them, but counting towards the
process FD limit).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13591

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
825d67fd by Andrew Bartlett at 2018-08-29T06:20:55Z
autobuild: Avoid subshell for tail -f invocation

This should mean one less process in the process tree, and less places to hold
FDs open.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13591

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Aug 29 08:20:55 CEST 2018 on sn-devel-144

- - - - -
81f4971f by Andreas Schneider at 2018-08-29T16:33:19Z
s4:torture: Improve torture_libsmbclient_init_context()

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
30ee1d3d by Andreas Schneider at 2018-08-29T16:33:19Z
selftest: Schedule libsmbclient tests against ad_dc

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7a7d95fc by Andreas Schneider at 2018-08-29T19:28:41Z
s4:torture: Add a test for listing shares

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Aug 29 21:28:41 CEST 2018 on sn-devel-144

- - - - -
99d6237a by Alexander Bokovoy at 2018-08-30T02:48:00Z
pidl/tests: fix ndr_push_init_ctx() usage

The code in libndr got rid of the second argument quite some time ago
(2010): f9ca9e46ad24036bf00cb361a6cef4b2e7e98d7d

However, pidl tests did include a code that passed two arguments.
Recently GCC started to fail to compile such code.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Aug 30 04:48:00 CEST 2018 on sn-devel-144

- - - - -
dc6040c1 by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-common: Add support for sock daemon to notify of successful startup

The daemon writes 0 into the specified file descriptor when it is up
and listening.  This can be used to avoid loops in clients that
attempt to connect until they succeed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
11ee92d1 by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-event: Add support to eventd for the startup notification FD

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e357b62f by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-daemon: Improve error handling consistency

Other errors free argv, so do it here too.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c446ae5e by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-daemon: Open eventd pipe earlier

The pipe will soon be needed earlier, so initialise it earlier.
Ensure the file descriptors are closed on error.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
62ec1ab1 by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-daemon: Wait for eventd to be ready before connecting

The current method of retrying the connection to eventd means that
messages get logged for each failure.

Instead, pass a pipe file descriptor to eventd and wait for it to
write 0 to the pipe to indicate that it is ready to accept client
connections.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
b430a1ac by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-daemon: Do not retry connection to eventd

Confirmation is now received from eventd that it is accepting
connections, so this is no longer needed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
03259348 by Martin Schwenke at 2018-08-30T02:48:56Z
ctdb-common: Fix CID 1414745 - Out-of-bounds access

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
7c361f48 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Restore dropped copyright attributions

Commit fa94a49dbbec4a65c368a533a534f952a9f147a7 accidentally dropped
some copyright attributions.  The original version of system_socket.c
was based on system_linux.c but many parts have been taking from
system_freebsd.c, which had these additional copyright attributions.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0927b382 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-tests: Add basic test to sanity check types in socket marshalling

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
172b87cb by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Initialise structures when declared

Instead of using ZERO_STRUCT().

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2ebb25df by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Factor out common ARP code

Finding the interface and the MAC address are obvious.  Might as well
set up the common parts of the destination address structure.

Continue to open the socket and find the MAC address first.  This
might seem odd because marshalling and other subsequent steps may
fail.  However, in the future this code might be optimised to open a
single socket to send ARPs for a list of addresses on each interface,
so don't change the logic.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
50a6d152 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Fix error handling when sending ARPs

There are numerous places in the code where errno can be lost causing
the wrong error to be printed by a caller.  Change ctdb_sys_send_arp()
to always return a useful errno on error instead of returning -1 and
sometimes having errno set correctly.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
39cfd511 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Separate ARP and IPv6 NA marshalling code

This can be tested separately.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
87088af6 by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-tests: Add tests for ARP and IPv6 NA marshalling

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e2a00fec by Martin Schwenke at 2018-08-30T02:48:57Z
ctdb-common: Be more careful with packet sizes

Ethernet packets must be at least 64 bytes.

For ARP the packet size was limited to 64 bytes.  This is probably OK
but the code might as well be a little more general.

For IPv6 NA there was no guarantee that the packet is at least 64
bytes.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
6b1e9a43 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Use struct ether_arp to avoid manual offset calculations

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
ca0db67d by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Clarify offset and packet length calculations

Calculate each offset from the beginning of the buffer and explicitly
use the sizes of structures.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
af5a42bf by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Set version more obviously in IPv6 NA packet

Version is the top 4 bits of this field.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a6789957 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Avoid single line multi-assignment

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d7d23e78 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Factor out TCP packet marshalling code

This can be tested separately.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a02cba1c by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-tests: Add tests for TCP packet marshalling

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8fcf1af5 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Avoid magic numbers when building TCP packets

Most packet sizes and offsets are multiples of 32-bit words.  The IPv6
payload length is in octets.  The IPv6 version is the top 4 bits of
the relevant field.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0beb16f3 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Don't modify a const argument

The current code might be slightly more efficient but
intentionally (although temporarily) modifying a const argument just
seems wrong.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f3a1f1e1 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Fix a bug in non-Linux (PCAP) TCP packet capturing

Captured packets include a link-layer header, which is considered in
the Linux code but not the PCAP code.  Also, the actual captured
length is in caplen, not len.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
cb4848e3 by Martin Schwenke at 2018-08-30T02:48:58Z
ctdb-common: Fix error handling when parsing TCP packets

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
028fdc12 by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-common: Clean up types/declarations in TCP socket reading

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e2ac3686 by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-common: Factor out TCP packet parsing code

This can be tested separately.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
43a20225 by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-tests: Extend TCP packet test to also do packet extraction

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
924a655b by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-common: Improve TCP packet size and offset calculations

The IPv4 check for short packets was strange.  It appeared to ensure
that the capture included everything up to and including the window
size.  The checksum field immediately follows the window size field,
so just ensure that the packet is large enough to contain everything
up to the start of the checksum.

Add a similar check for IPv6 packets.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
53ceac96 by Martin Schwenke at 2018-08-30T02:48:59Z
ctdb-common: Check the version field in IPv6 packets

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
58b8f2a3 by Martin Schwenke at 2018-08-30T05:50:04Z
ctdb-common: Clean up comments in TCP packet parsing

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Thu Aug 30 07:50:04 CEST 2018 on sn-devel-144

- - - - -
9f60a77e by Christof Schmitt at 2018-08-30T16:07:14Z
lib: Fix lstat check in directory_create_or_exist

The lstat check in directory_create_or_exist did not verify whether an
existing object is actually a directory. Also move the check to only
apply when mkdir returns EEXIST; this fixes CID 241930 Time of check
time of use.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
83f0ac5c by Christof Schmitt at 2018-08-30T19:19:31Z
torture: Add test for directory_create_or_exist

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Thu Aug 30 21:19:31 CEST 2018 on sn-devel-144

- - - - -
b7d77ce4 by Volker Lendecke at 2018-08-31T16:42:31Z
lib: Fix the build on FreeBSD

FreeBSD needs to explicitly #include <unistd.h> for geteuid() and close()

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Aug 31 18:42:31 CEST 2018 on sn-devel-144

- - - - -
44840ba5 by Ralph Boehme at 2018-08-31T20:22:22Z
vfs_delay_inject: adding delay to VFS calls

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3255822f by Ralph Boehme at 2018-08-31T20:22:22Z
s4:selftest: reformat smb2_s3only list

No change besides reformatting the list to one entry per line.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5508024a by Ralph Boehme at 2018-08-31T20:22:23Z
selftest: add a durable handle test with delayed disconnect

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b70b8503 by Ralph Boehme at 2018-08-31T20:22:23Z
s3:smbd: reorder tcon global record deletion and closing files of a tcon

As such, this doesn't change overall behaviour, but in case we ever add
semantics acting on tcon record changes via an API like
dbwrap_watch_send(), this will make a difference as it enforces
ordering.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8f6edcc1 by Ralph Boehme at 2018-08-31T20:22:23Z
s3:smbd: let session logoff close files and tcons before deleting the session

This avoids a race in durable handle reconnects if the reconnect comes
in while the old session is still in the tear-down phase.

The new session is supposed to rendezvous with and wait for destruction
of the old session, which is internally implemented with
dbwrap_watch_send() on the old session record.

If the old session deletes the session record before calling
file_close_user() which marks all file handles as disconnected, the
durable handle reconnect in the new session will fail as the records are
not yet marked as disconnected which is a prerequisite.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5d95f79f by Ralph Boehme at 2018-08-31T23:26:35Z
s3:smbd: add a comment stating that file_close_user() is redundant for SMB2

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13549

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Sep  1 01:26:35 CEST 2018 on sn-devel-144

- - - - -
cfb19e98 by Andrew Bartlett at 2018-09-03T01:22:19Z
selftest: Set NSS_WRAPPER_HOSTS when creating the trusts

Otherwise this relies on the order that tests run to cause the environment variable
to be left behind.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
85ef0590 by Andrew Bartlett at 2018-09-03T01:22:20Z
selftest: Set RESOLV_WRAPPER_CONF/RESOLV_WRAPPER_HOSTS when running dcpromo

Otherwise this relies on the order that tests run to cause the environment variable
to be left behind.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
ac30d7b6 by Andrew Bartlett at 2018-09-03T01:22:20Z
selftest: Remove excption for NSS_WRAPPER_HOSTS and RESOLV_WRAPPER_HOSTS

These must be set correctly for each command in provision also.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
45c0ec85 by Noel Power at 2018-09-03T01:22:20Z
python/samba: Fix py2/3 relative module import issue

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9688480d by Noel Power at 2018-09-03T01:22:20Z
python/samba: port open to io.open for PY2/PY3 compat

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
596db3a5 by Noel Power at 2018-09-03T01:22:20Z
python/samba: open files in binary mode as appropriate

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
27df0e81 by Noel Power at 2018-09-03T01:22:20Z
python/samba: port changes to allow samba.tests.dsdb_lock to work with PY3/PY2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
999b6db6 by Noel Power at 2018-09-03T01:22:20Z
s4/selftest: enable samba.tests.dsdb_lock for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
48f3801c by Noel Power at 2018-09-03T01:22:20Z
python/samba/tests: port samba.tests.unix for py3

L for long literal is no longer valid in Python3, also
long keyword has been removed. int and long have been unified
in python3. A duplicated test has been removed (for long and
int) only the int test remains now.
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5efb5ad9 by Noel Power at 2018-09-03T01:22:21Z
s4/selftest: enable samba.tests.dcerpc.unix for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0e53629f by Noel Power at 2018-09-03T01:22:21Z
s4/selftest/tests: enable samba.tests.samba_tool.timecmd for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
563c10ee by Noel Power at 2018-09-03T01:22:21Z
python/samba/tests: Use io.StringIO for py3 and StringIO.StrinIO for py2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
94c23ee0 by Noel Power at 2018-09-03T01:22:21Z
python/compat: use cStringIO instead of StringIO in compat.py

Using cStringIO for py2 seems to incur alot less problems and less changes
to the py2/py3 code then using StringIO.StringIO

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c6100d77 by Noel Power at 2018-09-03T01:22:21Z
python/samba: changes to make samba.tests.samba_tool.join run under py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bff4d80a by Noel Power at 2018-09-03T01:22:21Z
s4/selftest: Enable samba.tests.samba_tool.join for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
471163e8 by Noel Power at 2018-09-03T01:22:21Z
python/samba: changes needed for samba.tests.samba_tool.visualize PY2/PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e59ed0a3 by Noel Power at 2018-09-03T01:22:21Z
enable samba_tool.visualize_drs samba_tool.visualize_drs for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a701a253 by Noel Power at 2018-09-03T01:22:22Z
python/samba/tests: Port fsmo test to python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f0f1872e by Noel Power at 2018-09-03T01:22:22Z
s3/selftest: Enable samba.tests.samba_tool.fsmo for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
783d8565 by Noel Power at 2018-09-03T01:22:22Z
s4/selftest: enable samba.tests.samba_tool.processes for PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
210b950c by Noel Power at 2018-09-03T01:22:22Z
python/samba: Add some helper functions to encode/decode strings

Sometimes in PY3 we have variables whose content can be 'bytes' or
'str' and we can't be sure which. Generally this is because the
code variable to be used can be initialised (or reassigned) a value
from different api(s) or functions depending on complex conditions or
logic. Or another common case is in PY2 the variable is 'type <str>'
and in PY3 it is 'class <str>' but the function to use e.g. b64encode
requires 'bytes' in PY3. In such cases in PY3 it would be nice to avoid
excessive testing in the client code, in PY2 we would like to ensure the
code runs unchanged. In otherwords we wish to avoid in PY2 unnecessary
decode() calls which convert otherwise 'str' types to 'unicode'.
Mixing 'str' and 'unicode' can cause strange and unexpected problems,
sometimes we can't avoid this but it we should avoid if we can.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
638476f8 by Noel Power at 2018-09-03T01:22:22Z
python/samba: PY3 port for samba.tests.samba_tool.user_wdigest test

In addition to the attributes that caused some issues specifically
with the test some other ldb.bytes objects (those seen to be
used as strings) have been adjusted (with str()) to ensure they should
work correct in PY3.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d29fdb02 by Noel Power at 2018-09-03T01:22:22Z
s4/selftest: enable samba.tests.samba_tool.user_wdigest for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
765b1762 by Noel Power at 2018-09-03T01:22:22Z
s4/selftest: enable samba.tests.samba_tool.user_check_password_script py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dd1fce7c by Noel Power at 2018-09-03T01:22:22Z
python/sambe/tests/samba_tool: PY3 port for tests.samba_tool.group

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a167c449 by Noel Power at 2018-09-03T01:22:23Z
s4/selftest: enable samba.tests.group for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4764ac05 by Noel Power at 2018-09-03T01:22:23Z
s4/selftest: enable samba.tests.samba_tool.ou for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
683d51fb by Noel Power at 2018-09-03T01:22:23Z
python/samba/netcmd: changes for samab.tests.samba_tool.computer

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
84c890b2 by Noel Power at 2018-09-03T01:22:23Z
python/samba/provision: remove use of str() func for binary data

Python 2 code works with str(policy["nTSecurityDescriptor"]) however
this cannot work with Python 3. One could argue even the str method
doesn't make sense at all (returning a string) for data.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7a2df705 by Noel Power at 2018-09-03T01:22:23Z
s4/selftest: enable samba.tests.samba_tool.ntacl for Py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
252950ac by Noel Power at 2018-09-03T01:22:23Z
python/samba/netcmd: Fix password usage for py2/py3 compatability

getpass returns str (e.g. bytes) in python2 and str (unicode) in
py3. Adapt code to so we don't do illegal things (like try and decode)
a string in python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4188a02b by Noel Power at 2018-09-03T01:22:23Z
s4/selftest: Enable samba.tests.samba_tool.provision_password_check for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f86693c8 by Noel Power at 2018-09-03T01:22:23Z
python/samba/tests: Py2/Py2 enable samba.tests.samba_tool.help to run

Ensure bytes output is converted to text type for PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9bded41d by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: enable samba.tests.samba_tool.help for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ffdbc4aa by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: Enable samba.tests.samba_tool.sites for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f304a273 by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: enable samba.tests.samba_tool.dnscmd for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
945359bc by Noel Power at 2018-09-03T01:22:24Z
selftest: Update known fail with py3 variant of samba.tests.dcerpc.dnsserver

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
179cd3db by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: enable samba.tests.dcerpc.dnsserver for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b3a5dac3 by Noel Power at 2018-09-03T01:22:24Z
s4/selftest: enable samba.tests.dcerpc.dnsserver(ad_dc) for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c958a043 by Noel Power at 2018-09-03T01:22:24Z
python/samba/tests: ensure byte content (not strings)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a467dc2a by Noel Power at 2018-09-03T01:22:25Z
python/samba/test/dcerpc: Py3 port of samba.tests.dcerpc.raw_protocol

Port code to allow this test run with either py2 or py3
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f5726ddc by Noel Power at 2018-09-03T01:22:25Z
s4/selftest: enable samba.tests.dcerpc.raw_protocol for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9df799d1 by Noel Power at 2018-09-03T01:22:25Z
python/samba/tests: PY2/PY3 required changes for samba.tests.samba_tool.user

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c6fd15e2 by Noel Power at 2018-09-03T01:22:25Z
python/samba/tests: Adjust teardown so second run of tests succeeds

A second run of this test (e.g. with --extra-python) or even
an manual re-run and the cache is still there.
use of '--cache-ldb-initialize'  will fail and so with the test
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
17290a37 by Noel Power at 2018-09-03T01:22:25Z
s4/selftest: enable samba.tests.samba_tool.user for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
82408fdc by Noel Power at 2018-09-03T01:22:25Z
python/samba/netcmd: PY2/PY3 changes required for user_virtualCryptSHA test

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9eb26065 by Noel Power at 2018-09-03T01:22:25Z
s4/selftest: enable samba.tests.samba_tool.user_virtualCryptSHA for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a740a613 by Andrew Bartlett at 2018-09-03T01:22:25Z
selftest: Prepare to allow override of lockout duration in password_lockout tests

This will make it easier to avoid flapping tests.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
74357bf3 by Andrew Bartlett at 2018-09-03T04:14:55Z
selftest: Split up password_lockout into tests with and without a call to sleep()

This means we can have a long observation window for many of the tests and
so make them much more reliable.  Many of these cause frustrating flapping
failures in our CI systems.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Sep  3 06:14:55 CEST 2018 on sn-devel-144

- - - - -
6d3d9a85 by Martin Schwenke at 2018-09-03T08:52:10Z
ctdb-daemon: Log complete eventd startup command

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13592

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
b5453bc2 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-daemon: Drop incorrect log message

The message is incorrect because the actual failure was loading the
config file.  Instead of fixing the message, drop it because
ctdb_config_load() already logs the failure.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
421d828f by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Fix log message for conf option with unknown section

This covers both options that appear before a section and options in
unknown sections.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
ebb28c57 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Log a message for unknown conf option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a017d318 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Log a message when an invalid conf value is encountered

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f1084400 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Avoid ENOENT for unknown conf type tags

Only use ENOENT for missing configuration file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
920ed66b by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Avoid ENOENT for unknown conf options

Only use ENOENT for missing configuration file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4f1727fe by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-common: Process the whole config file even if an error occurs

At the moment multiple errors will be encountered one at a time, on
each load or validate.  Instead, allow all configuration errors to
printed in a single pass.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13589

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
eed738a3 by Martin Schwenke at 2018-09-03T08:52:11Z
ctdb-tests: If bin/ isn't in ctdb/ then look one level higher

CTDB's test suite doesn't work from a top-level compile.  The first
step to fixing this is to correctly locate the bin/ directory.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8aacde3c by Martin Schwenke at 2018-09-03T08:52:12Z
ctdb-tests: Use known install paths in local daemon tests

The in-tree local daemons tests don't work from a top-level Samba
compile.  The simple test suite was the first test suite and things
have generally worked, so it has been slow to adopt general test
infrastructure changes.

Instead of re-calculating script and helper locations, use the paths
from script_install_paths.sh.  The bin/ directory is already added to
PATH in common.sh, so don't add it here.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
30eb2881 by Martin Schwenke at 2018-09-03T12:04:00Z
ctdb-tests: Don't run valgrind or other tracing in simple_test_command()

This function is used to run a extra command to check a result.  This
command is usually a script (often a stub) or an external command, so
no need to trace it with valgrind or whatever else might be specified.
In the worst case the command being run is a shell function, which
valgrind won't be able to find.

There is little use running the event script tests under valgrind.
However, when the whole test suite is being run under valgrind then it
should work.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Sep  3 14:04:00 CEST 2018 on sn-devel-144

- - - - -
0bd109b7 by Volker Lendecke at 2018-09-03T16:44:23Z
smbd: Fix a memleak in async search ask sharemode

fetch_share_mode_unlocked_parser() takes a "struct
fetch_share_mode_unlocked_state *" as
"private_data". fetch_share_mode_send() used a talloc_zero'ed "struct
share_mode_lock". This lead to the parser putting a "struct
share_mode_lock on the NULL talloc_context where nobody really picked it
up.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13602
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
89f9c163 by Volker Lendecke at 2018-09-03T16:44:23Z
dbwrap_tool: Simplify dbwrap_tool_erase

That's what dbwrap_wipe is for :-)

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
46819e26 by Volker Lendecke at 2018-09-03T16:44:24Z
dbwrap_tool: Simplify listkey_fn

To me dbwrap_record_get_key(rec).dsize just looks a bit ugly

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
02d44842 by Volker Lendecke at 2018-09-03T16:44:24Z
dbwrap_tool: Simplify persistent/non-persistent check

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
0ce26c75 by Volker Lendecke at 2018-09-03T16:44:24Z
dbwrap_tool: Avoid an unnecessary "else"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
05e618cb by Volker Lendecke at 2018-09-03T19:38:40Z
dbwrap_tool: We don't do "listwatchers" anymore

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Mon Sep  3 21:38:40 CEST 2018 on sn-devel-144

- - - - -
dffc182c by Stefan Metzmacher at 2018-09-04T00:31:26Z
s4:torture/rpc/netlogon: assert that cli_credentials_get_{workstation,password} don't return NULL

This is better that generating a segfault while dereferencing a NULL
pointer later.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d5dd8fdc by Stefan Metzmacher at 2018-09-04T00:31:26Z
s4:torture/rpc/netlogon: verify the trusted domains output of LogonGetDomainInfo()

This makes sure we don't treat trusted domains in the same way we treat
our primary domain.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f5f96f55 by Stefan Metzmacher at 2018-09-04T00:31:26Z
dsdb/util_trusts: domain_dn is an input parameter of dsdb_trust_crossref_tdo_info()

We should not overwrite it within the function.
Currently it doesn't matter as we don't have multiple domains
within our forest, but that will change in future.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c1b0ac95 by Stefan Metzmacher at 2018-09-04T00:31:26Z
dsdb:util_trusts: add dsdb_trust_local_tdo_info() helper function

This is similar to dsdb_trust_xref_tdo_info(), but will also work
if we ever support more than one domain in our forest.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0e442e09 by Stefan Metzmacher at 2018-09-04T00:31:27Z
s4:dsdb/common: add samdb_domain_guid() helper function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
61333f77 by Stefan Metzmacher at 2018-09-04T00:31:27Z
s4:rpc_server/netlogon: use samdb_domain_guid()/dsdb_trust_local_tdo_info() to build our netr_OneDomainInfo values

The logic for constructing the values for our own primary domain differs
from the values of trusted domains. In order to make the code easier to
understand we have a new fill_our_one_domain_info() helper that
only takes care of our primary domain.

The cleanup for the trust case will follow in a separate commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ef0b489a by Stefan Metzmacher at 2018-09-04T00:31:27Z
s4:rpc_server/netlogon: make use of talloc_zero_array() for the netr_OneDomainInfo array

It's much safer than having uninitialized memory when we hit an error
case.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2099add0 by Stefan Metzmacher at 2018-09-04T00:31:27Z
s4:rpc_server/netlogon: don't treet trusted domains as primary in LogonGetDomainInfo()

We need to handle trusted domains differently than our primary
domain. The most important part is that we don't return
NETR_TRUST_FLAG_PRIMARY for them.

NETR_TRUST_FLAG_{INBOUND,OUTBOUND,IN_FOREST} are the relavant flags
for trusts.

This is an example of what Windows returns in a complex trust
environment:

     netr_LogonGetDomainInfo: struct netr_LogonGetDomainInfo
        out: struct netr_LogonGetDomainInfo
            return_authenticator     : *
                return_authenticator: struct netr_Authenticator
                    cred: struct netr_Credential
                        data                     : f48b51ff12ff8c6c
                    timestamp                : Tue Aug 28 22:59:03 2018 CEST
            info                     : *
                info                     : union netr_DomainInfo(case 1)
                domain_info              : *
                    domain_info: struct netr_DomainInformation
                        primary_domain: struct netr_OneDomainInfo
                            domainname: struct lsa_StringLarge
                                length                   : 0x0014 (20)
                                size                     : 0x0016 (22)
                                string                   : *
                                    string                   : 'W2012R2-L4'
                            dns_domainname: struct lsa_StringLarge
                                length                   : 0x0020 (32)
                                size                     : 0x0022 (34)
                                string                   : *
                                    string                   : 'w2012r2-l4.base.'
                            dns_forestname: struct lsa_StringLarge
                                length                   : 0x0020 (32)
                                size                     : 0x0022 (34)
                                string                   : *
                                    string                   : 'w2012r2-l4.base.'
                            domain_guid              : 0a133c91-8eac-4df0-96ac-ede69044a38b
                            domain_sid               : *
                                domain_sid               : S-1-5-21-2930975464-1937418634-1288008815
                            trust_extension: struct netr_trust_extension_container
                                length                   : 0x0000 (0)
                                size                     : 0x0000 (0)
                                info                     : NULL
                            dummy_string2: struct lsa_StringLarge
                                length                   : 0x0000 (0)
                                size                     : 0x0000 (0)
                                string                   : NULL
                            dummy_string3: struct lsa_StringLarge
                                length                   : 0x0000 (0)
                                size                     : 0x0000 (0)
                                string                   : NULL
                            dummy_string4: struct lsa_StringLarge
                                length                   : 0x0000 (0)
                                size                     : 0x0000 (0)
                                string                   : NULL
                            dummy_long1              : 0x00000000 (0)
                            dummy_long2              : 0x00000000 (0)
                            dummy_long3              : 0x00000000 (0)
                            dummy_long4              : 0x00000000 (0)
                        trusted_domain_count     : 0x00000006 (6)
                        trusted_domains          : *
                            trusted_domains: ARRAY(6)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x000e (14)
                                        size                     : 0x0010 (16)
                                        string                   : *
                                            string                   : 'FREEIPA'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x0018 (24)
                                        size                     : 0x001a (26)
                                        string                   : *
                                            string                   : 'freeipa.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 00000000-0000-0000-0000-000000000000
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-429948374-2562621466-335716826
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000022 (34)
                                                       0: NETR_TRUST_FLAG_IN_FOREST
                                                       1: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       1: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000000 (0)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000008 (8)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       1: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x0016 (22)
                                        size                     : 0x0018 (24)
                                        string                   : *
                                            string                   : 'S1-W2012-L4'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x0036 (54)
                                        size                     : 0x0038 (56)
                                        string                   : *
                                            string                   : 's1-w2012-l4.w2012r2-l4.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : afe7fbde-af82-46cf-88a2-2df6920fc33e
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-1368093395-3821428921-3924672915
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000023 (35)
                                                       1: NETR_TRUST_FLAG_IN_FOREST
                                                       1: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       1: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000004 (4)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000020 (32)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       0: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       1: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x0006 (6)
                                        size                     : 0x0008 (8)
                                        string                   : *
                                            string                   : 'BLA'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x0010 (16)
                                        size                     : 0x0012 (18)
                                        string                   : *
                                            string                   : 'bla.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 00000000-0000-0000-0000-000000000000
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-4053568372-2049667917-3384589010
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000022 (34)
                                                       0: NETR_TRUST_FLAG_IN_FOREST
                                                       1: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       1: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000000 (0)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000008 (8)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       1: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x000c (12)
                                        size                     : 0x000e (14)
                                        string                   : *
                                            string                   : 'S4XDOM'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x0016 (22)
                                        size                     : 0x0018 (24)
                                        string                   : *
                                            string                   : 's4xdom.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 00000000-0000-0000-0000-000000000000
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-313966788-4060240134-2249344781
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000022 (34)
                                                       0: NETR_TRUST_FLAG_IN_FOREST
                                                       1: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       1: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000000 (0)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000008 (8)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       1: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x0014 (20)
                                        size                     : 0x0016 (22)
                                        string                   : *
                                            string                   : 'W2012R2-L4'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x001e (30)
                                        size                     : 0x0020 (32)
                                        string                   : *
                                            string                   : 'w2012r2-l4.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 0a133c91-8eac-4df0-96ac-ede69044a38b
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-2930975464-1937418634-1288008815
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x0000001d (29)
                                                       1: NETR_TRUST_FLAG_IN_FOREST
                                                       0: NETR_TRUST_FLAG_OUTBOUND
                                                       1: NETR_TRUST_FLAG_TREEROOT
                                                       1: NETR_TRUST_FLAG_PRIMARY
                                                       1: NETR_TRUST_FLAG_NATIVE
                                                       0: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000000 (0)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000000 (0)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       0: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                                trusted_domains: struct netr_OneDomainInfo
                                    domainname: struct lsa_StringLarge
                                        length                   : 0x0016 (22)
                                        size                     : 0x0018 (24)
                                        string                   : *
                                            string                   : 'S2-W2012-L4'
                                    dns_domainname: struct lsa_StringLarge
                                        length                   : 0x004e (78)
                                        size                     : 0x0050 (80)
                                        string                   : *
                                            string                   : 's2-w2012-l4.s1-w2012-l4.w2012r2-l4.base'
                                    dns_forestname: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    domain_guid              : 29daace6-cded-4ce3-a754-7482a4d9127c
                                    domain_sid               : *
                                        domain_sid               : S-1-5-21-167342819-981449877-2130266853
                                    trust_extension: struct netr_trust_extension_container
                                        length                   : 0x0010 (16)
                                        size                     : 0x0010 (16)
                                        info                     : *
                                            info: struct netr_trust_extension
                                                length                   : 0x00000008 (8)
                                                dummy                    : 0x00000000 (0)
                                                size                     : 0x00000008 (8)
                                                flags                    : 0x00000001 (1)
                                                       1: NETR_TRUST_FLAG_IN_FOREST
                                                       0: NETR_TRUST_FLAG_OUTBOUND
                                                       0: NETR_TRUST_FLAG_TREEROOT
                                                       0: NETR_TRUST_FLAG_PRIMARY
                                                       0: NETR_TRUST_FLAG_NATIVE
                                                       0: NETR_TRUST_FLAG_INBOUND
                                                       0: NETR_TRUST_FLAG_MIT_KRB5
                                                       0: NETR_TRUST_FLAG_AES
                                                parent_index             : 0x00000001 (1)
                                                trust_type               : LSA_TRUST_TYPE_UPLEVEL (2)
                                                trust_attributes         : 0x00000000 (0)
                                                       0: LSA_TRUST_ATTRIBUTE_NON_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_UPLEVEL_ONLY
                                                       0: LSA_TRUST_ATTRIBUTE_QUARANTINED_DOMAIN
                                                       0: LSA_TRUST_ATTRIBUTE_FOREST_TRANSITIVE
                                                       0: LSA_TRUST_ATTRIBUTE_CROSS_ORGANIZATION
                                                       0: LSA_TRUST_ATTRIBUTE_WITHIN_FOREST
                                                       0: LSA_TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL
                                                       0: LSA_TRUST_ATTRIBUTE_USES_RC4_ENCRYPTION
                                    dummy_string2: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string3: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_string4: struct lsa_StringLarge
                                        length                   : 0x0000 (0)
                                        size                     : 0x0000 (0)
                                        string                   : NULL
                                    dummy_long1              : 0x00000000 (0)
                                    dummy_long2              : 0x00000000 (0)
                                    dummy_long3              : 0x00000000 (0)
                                    dummy_long4              : 0x00000000 (0)
                        lsa_policy: struct netr_LsaPolicyInformation
                            policy_size              : 0x00000000 (0)
                            policy                   : NULL
                        dns_hostname: struct lsa_StringLarge
                            length                   : 0x0036 (54)
                            size                     : 0x0038 (56)
                            string                   : *
                                string                   : 'torturetest.w2012r2-l4.base'
                        dummy_string2: struct lsa_StringLarge
                            length                   : 0x0000 (0)
                            size                     : 0x0000 (0)
                            string                   : NULL
                        dummy_string3: struct lsa_StringLarge
                            length                   : 0x0000 (0)
                            size                     : 0x0000 (0)
                            string                   : NULL
                        dummy_string4: struct lsa_StringLarge
                            length                   : 0x0000 (0)
                            size                     : 0x0000 (0)
                            string                   : NULL
                        workstation_flags        : 0x00000003 (3)
                               1: NETR_WS_FLAG_HANDLES_INBOUND_TRUSTS
                               1: NETR_WS_FLAG_HANDLES_SPN_UPDATE
                        supported_enc_types      : 0x0000001f (31)
                               1: KERB_ENCTYPE_DES_CBC_CRC
                               1: KERB_ENCTYPE_DES_CBC_MD5
                               1: KERB_ENCTYPE_RC4_HMAC_MD5
                               1: KERB_ENCTYPE_AES128_CTS_HMAC_SHA1_96
                               1: KERB_ENCTYPE_AES256_CTS_HMAC_SHA1_96
                               0: KERB_ENCTYPE_FAST_SUPPORTED
                               0: KERB_ENCTYPE_COMPOUND_IDENTITY_SUPPORTED
                               0: KERB_ENCTYPE_CLAIMS_SUPPORTED
                               0: KERB_ENCTYPE_RESOURCE_SID_COMPRESSION_DISABLED
                        dummy_long3              : 0x00000000 (0)
                        dummy_long4              : 0x00000000 (0)
            result                   : NT_STATUS_OK

Best viewed with: git show --histogram -w

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11517

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
38e97f8b by Andreas Schneider at 2018-09-04T00:31:27Z
waf: Check for -fstack-protect-strong support

The -fstack-protector* flags are compiler only flags, don't pass them to
the linker.

https://developers.redhat.com/blog/2018/03/21/compiler-and-linker-flags-gcc/

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13601

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fc4df251 by Andreas Schneider at 2018-09-04T00:31:27Z
waf: Add -fstack-clash-protection

https://developers.redhat.com/blog/2018/03/21/compiler-and-linker-flags-gcc/

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13601

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fb047b3d by Robert Scheck at 2018-09-04T00:31:27Z
Update Samba URLs from http:// to https://

Reviewed-by: Volker Lendecke <Volker.Lendecke at SerNet.DE>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f98f8e3e by Jeremy Allison at 2018-09-04T00:31:27Z
s4: torture: Ensure all notify tests use separate directories.

Makes it much easier to find tests that error out whilst
leaving directories behind.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1799633c by Jeremy Allison at 2018-09-04T00:31:27Z
s4: torture: Ensure we can't exit without deleting our directory.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c86f6c29 by Jeremy Allison at 2018-09-04T03:23:57Z
s4: torture: Ensure we close the handle on the correct tree-id.

Otherwise we leave a directory behind on exit.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Sep  4 05:23:57 CEST 2018 on sn-devel-144

- - - - -
7356e814 by Stefan Metzmacher at 2018-09-04T08:45:10Z
s3:vfs: fix valgrind warning in SMB_VFS_{PREAD,PWRITE,FSYNC}_RECV()

tevent_req_received() destroys 'state', so we need helper variables
to hold the return value.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Tue Sep  4 10:45:10 CEST 2018 on sn-devel-144

- - - - -
80324ff6 by Volker Lendecke at 2018-09-04T22:48:17Z
examples: Fix mingw64 detection

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7df50529 by Alexander Bokovoy at 2018-09-04T22:48:17Z
s4:selftest: test kinit with the interdomain trust user account

To test it, add a blackbox test that ensures we pass a keytab-based
authentication with the trust user account for a trusted domain.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
39c281a2 by Stefan Metzmacher at 2018-09-04T22:48:17Z
samba-tool: add virtualKerberosSalt attribute to 'user getpassword/syncpasswords'

This might be useful for someone, but at least it's very useful for
tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8526feb1 by Stefan Metzmacher at 2018-09-04T22:48:17Z
testprogs/blackbox: add testit[_expect_failure]_grep() to subunit.sh

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1b31fa62 by Stefan Metzmacher at 2018-09-04T22:48:17Z
testprogs/blackbox: let test_trust_user_account.sh check the correct kerberos salt

This demonstrates the bug we currently have.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f3e349be by Alexander Bokovoy at 2018-09-05T01:57:22Z
krb5-samba: interdomain trust uses different salt principal

Salt principal for the interdomain trust is krbtgt/DOMAIN at REALM where
DOMAIN is the sAMAccountName without the dollar sign ($)

The salt principal for the BLA$ user object was generated wrong.

dn: CN=bla.base,CN=System,DC=w4edom-l4,DC=base
securityIdentifier: S-1-5-21-4053568372-2049667917-3384589010
trustDirection: 3
trustPartner: bla.base
trustPosixOffset: -2147483648
trustType: 2
trustAttributes: 8
flatName: BLA

dn: CN=BLA$,CN=Users,DC=w4edom-l4,DC=base
userAccountControl: 2080
primaryGroupID: 513
objectSid: S-1-5-21-278041429-3399921908-1452754838-1597
accountExpires: 9223372036854775807
sAMAccountName: BLA$
sAMAccountType: 805306370
pwdLastSet: 131485652467995000

The salt stored by Windows in the package_PrimaryKerberosBlob
(within supplementalCredentials) seems to be
'W4EDOM-L4.BASEkrbtgtBLA' for the above trust
and Samba stores 'W4EDOM-L4.BASEBLA$'.

While the salt used when building the keys from
trustAuthOutgoing/trustAuthIncoming is
'W4EDOM-L4.BASEkrbtgtBLA.BASE', which we handle correct.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13539

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep  5 03:57:22 CEST 2018 on sn-devel-144

- - - - -
8077f462 by Thomas Nagy at 2018-09-05T04:37:21Z
build:wafsamba: Build on waf 1.9

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5d997861 by Thomas Nagy at 2018-09-05T04:37:21Z
build:wafsamba: Update dist/distcheck commands

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9aa8f2ba by Thomas Nagy at 2018-09-05T04:37:21Z
build:wafsamba: detail where we are processing the autobuild

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
59673353 by Thomas Nagy at 2018-09-05T04:37:21Z
script/autobuild: use --out instead of -b when calling configure

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
48cf9ccd by Thomas Nagy at 2018-09-05T04:37:21Z
selftest/tests.py: Update path to waflib

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cbc65346 by Thomas Nagy at 2018-09-05T04:37:22Z
thirdparty:waf: New files for waf 1.9.10

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
247d12c4 by Thomas Nagy at 2018-09-05T04:37:22Z
build:wafsamba: Ignore cfg_file absolute paths differences

Due to build variants, cfg_file paths are written as absolute paths.

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
faef2750 by Thomas Nagy at 2018-09-05T04:37:22Z
build:wafsamba: Remove unnecessary parameters to cmd_and_log

Signed-off-by: Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4e65b33c by Alexander Bokovoy at 2018-09-05T04:37:22Z
third_party:waf: update to upstream 2.0.4 release

Update third_party/waf/ to 2.0.4 to bring us closer to Python 3

This change requires a number of changes in buildtools/ too.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
65074d89 by Alexander Bokovoy at 2018-09-05T04:37:22Z
ctdb/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
efa94eb3 by Alexander Bokovoy at 2018-09-05T04:37:22Z
docs-xml/wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
18c3598b by Alexander Bokovoy at 2018-09-05T04:37:22Z
dynconfig/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b999fd3c by Alexander Bokovoy at 2018-09-05T04:37:22Z
lib/crypto/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b0a12c81 by Alexander Bokovoy at 2018-09-05T04:37:22Z
lib/crypto/wscript_configure: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0c703f1d by Alexander Bokovoy at 2018-09-05T04:37:22Z
lib/ldb/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ea578830 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/replace/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
62bb5734 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/talloc/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
15288c38 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/tdb/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d4cef3b1 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/tevent/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5f577140 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/util/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e6761832 by Alexander Bokovoy at 2018-09-05T04:37:23Z
lib/util/wscript_configure: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8a741163 by Alexander Bokovoy at 2018-09-05T04:37:23Z
nsswitch/wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e1e5b57f by Alexander Bokovoy at 2018-09-05T04:37:23Z
packaging/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7f6ce8f8 by Alexander Bokovoy at 2018-09-05T04:37:23Z
pidl/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fa1ca713 by Alexander Bokovoy at 2018-09-05T04:37:23Z
python/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0de67cf1 by Alexander Bokovoy at 2018-09-05T04:37:23Z
selftest/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c898f18f by Alexander Bokovoy at 2018-09-05T04:37:23Z
source3/build/charset.py: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
91e099d6 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source3/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8b5ad021 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source3/wscript_configure_system_ncurses: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
029ac7d6 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source4/dsdb/samdb/ldb_modules/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
055aae90 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source4/heimdal_build/wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c797e921 by Alexander Bokovoy at 2018-09-05T04:37:24Z
source4/heimdal_build/wscript_configure: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0c423a3a by Alexander Bokovoy at 2018-09-05T04:37:24Z
source4/lib/tls/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ca5008a4 by Alexander Bokovoy at 2018-09-05T04:37:24Z
testsuite/headers/wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ba03d120 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/aesni-intel/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3e71f06a by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/cmocka/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2b21e67e by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/nss_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2847a382 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/pam_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7e2deed7 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/popt/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4c441530 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/resolv_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
051e7e74 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/socket_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f14b8cb0 by Alexander Bokovoy at 2018-09-05T04:37:24Z
third_party/uid_wrapper/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0119a879 by Alexander Bokovoy at 2018-09-05T04:37:25Z
third_party/wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c73b7795 by Alexander Bokovoy at 2018-09-05T04:37:25Z
wscript: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1dc80eb1 by Alexander Bokovoy at 2018-09-05T04:37:25Z
wscript_build: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e58ca30b by Alexander Bokovoy at 2018-09-05T04:37:25Z
wscript_build_embedded_heimdal: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ef4b2d45 by Alexander Bokovoy at 2018-09-05T04:37:25Z
wscript_configure_system_mitkrb5: update to handle waf 2.0.4

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c5ab9ead by Alexander Bokovoy at 2018-09-05T04:37:25Z
heimdal wscript changes

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
79c4ba26 by Alexander Bokovoy at 2018-09-05T04:37:25Z
auth/wscript: fix options use

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3b7dfc51 by Alexander Bokovoy at 2018-09-05T04:37:25Z
third_party/waf: upgrade to waf 2.0.8

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7eab91a0 by Alexander Bokovoy at 2018-09-05T04:37:25Z
buildtools/wafsamba: adopt to waf 2.0.8

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
69c655fe by Alexander Bokovoy at 2018-09-05T04:37:25Z
buildtools/wafsamba: reduce imports

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3fc47866 by Alexander Bokovoy at 2018-09-05T04:37:25Z
buildtools/wafsamba: add install_dir to build context

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
944fce25 by Alexander Bokovoy at 2018-09-05T04:37:25Z
auth/wscript: import from waflib

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d00ba400 by Alexander Bokovoy at 2018-09-05T04:37:25Z
nsswitch/libwbclient/wscript: import from waflib

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9f2f5b4d by Alexander Bokovoy at 2018-09-05T04:37:26Z
source3/libsmb/wscript: remove unneeded import

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7aaa1e68 by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba/samba_abi: always_run helper was deprecated in waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
175be937 by Alexander Bokovoy at 2018-09-05T04:37:26Z
ctdb/wscript: adopt to waf-2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2e401d24 by Alexander Bokovoy at 2018-09-05T04:37:26Z
wscript: adopt to waf-2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
850ceec8 by Thomas Nagy at 2018-09-05T04:37:26Z
buildtools/wafsamba: compile asn1 files by adding missing code from compat15

Signed-off-by:    Thomas Nagy <tnagy at waf.io>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1fdcbd06 by Alexander Bokovoy at 2018-09-05T04:37:26Z
wafsamba: use correct context for APPNAME

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2a63619c by Alexander Bokovoy at 2018-09-05T04:37:26Z
waf heimdal: use absolute path to compile_et

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
35ebfd3d by Alexander Bokovoy at 2018-09-05T04:37:26Z
selftest/wscript: handle lists in environmental variables in waf

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cdda0d90 by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba: change SAMBA_BUILD_ENV to use bldnode.abspath()

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0fdba4be by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba: use top for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
699977a2 by Alexander Bokovoy at 2018-09-05T04:37:26Z
wscript: adopt to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5c3d31eb by Alexander Bokovoy at 2018-09-05T04:37:26Z
cdtb/wscript: use top and out for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fb43723b by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba: use top and out for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
aebcd698 by Alexander Bokovoy at 2018-09-05T04:37:26Z
buildtools/wafsamba: crosscompile should use Utils.subprocess in waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1d25ae58 by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: use context instead of options for cross-compile checks for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
77f3a138 by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: use cflags instead of ccflags for waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
31f8945a by Alexander Bokovoy at 2018-09-05T04:37:27Z
wafsamba: install Python modules back to bin/python, not bin/python_modules

Partially revert 80fce353e740c793619005ac102ab07fb5e7d280 which started
installing generated Python modules into python_modules directory back
in 2013. This, unfortunately, does not work anymore as Python gets quite
confused with our setup even when both bin/python and bin/python_modules
directories are part of sys.path.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
75d5bcb4 by Alexander Bokovoy at 2018-09-05T04:37:27Z
selftest/wscript: properly handle env.cwd which is a list, not a string

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2a3fcdf1 by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: generate build options output with waf 2.0

With WAF 2.0 we get all defines in environment at the same level.
Fix build options source code generator to handle this.

I felt uneasy at filtering out some defines so instead the code
is mangling generic defines to be correct for C compiler by
replacing '-', '.', and '()' with an underscore ('_').

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7944ed6e by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: port stale_files to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1efe6890 by Alexander Bokovoy at 2018-09-05T04:37:27Z
wscript: port build_system_mitkrb5 to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
72a9e6d1 by Alexander Bokovoy at 2018-09-05T04:37:27Z
wscript: port build_system_heimdal to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fdd89fef by Alexander Bokovoy at 2018-09-05T04:37:27Z
selftest/tests.py: update to support waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0a9d98ba by Alexander Bokovoy at 2018-09-05T04:37:27Z
ctdb/wscript: rework how version number is retrieved

Using default context functions before waf initialization occured
is prone to error. Postpone calling samba_version.* code until we
got default context initialized.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f495f64e by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: remove ENFORCE_GROUP_ORDERING

ENFORCE_GROUP_ORDERING is not needed with waf 2.0 anymore

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4c7c10b2 by Alexander Bokovoy at 2018-09-05T04:37:27Z
lib/audit_logging: update to waf 2.0

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6fc9f1a8 by Alexander Bokovoy at 2018-09-05T04:37:27Z
lib/mscat: fix logging in wscript

- - - - -
72c3ff9a by Alexander Bokovoy at 2018-09-05T04:37:27Z
buildtools/wafsamba: use CACHE_SUFFIX instead of a hard-coded name

waflib.Build provides CACHE_SUFFIX constant to append to the target
name. We have a reference to samba-specific cache suffix (.cache.py)
while original WAF uses _cache.py as a cache suffix since 2011 (see
commit 44a967e326cc2e670a31b3712e4763b72d65e81b in WAF project code).

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
67ed1ea6 by Alexander Bokovoy at 2018-09-05T04:37:28Z
script/autobuild: re-use CACHE_SUFFIX from waflib

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8f022a0b by Alexander Bokovoy at 2018-09-05T04:37:28Z
script/autobuild: Fix formatting in send_email

Commit cb40e2bbc8a34a1ec3584ab585c5bf44c037ef0e introduced a print
statement with a broken formatting. Reported by pylint.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8de348e9 by Andrew Bartlett at 2018-09-05T07:29:39Z
third_party: Import exact files from waf-2.0.8/waflib

wget https://waf.io/waf-2.0.8.tar.bz2
tar -xf waf-2.0.8.tar.bz2
rsync -a waf-2.0.8/waflib/ third_party/waf/waflib/

The previous import was damaged by auto-strip/correct of whitespace
and had other small corrections.

Check with git show -w.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep  5 09:29:39 CEST 2018 on sn-devel-144

- - - - -
630cc6e6 by Andrew Bartlett at 2018-09-05T09:42:23Z
torture: Add tests to prove that kinit to an SPN is not allowed (unless it is also a UPN)

The krb5.kdc.canon testsuite has been updated to pass against Windows
Server 1709 in four modes:

* A normal user
* A user with a UPN
* A user with an SPN (machine account)
* A user with an SPN as the UPN

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
364c13ac by Andrew Bartlett at 2018-09-05T09:42:24Z
selftest/samba4.blackbox.export.keytab: Remove stray exit 0 and so run cleanup

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
a6182bd9 by Andrew Bartlett at 2018-09-05T09:42:24Z
Revert "s4/heimdal: allow SPNs in AS-REQ"

This reverts commit 20dc68050df7b1b0c9d06f8251183a0a6283fcaf.

Tests (the krb5.kdc testsuite) show this behaviour is incorrect.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
71ba7cb9 by Andrew Bartlett at 2018-09-05T09:42:24Z
selftest: Add new test to run krb5.kdc.canon against a user with an SPN for a UPN

The failures in this test compared with Windows Server 1709 are added to
knownfail.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3e5ad202 by Andrew Bartlett at 2018-09-05T09:42:25Z
selftest/samba4.blackbox.export.keytab: Update to use a principal with SPN as UPN

The ability the kinit with an SPN (not also being a UPN) has gone away as
windows doesn't offer this functionality.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
9e1ba904 by Andrew Bartlett at 2018-09-05T09:42:26Z
torture: Confirm that this element of the krb5.kdc test does not pass against Windows

This should be fixed, but in the meantime add clue to avoid regressions on
bug 11539.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
f7c409c4 by Andrew Bartlett at 2018-09-05T09:42:26Z
torture krb5.kdc.canon: Correct principal being checked in TEST_AS_REQ_SELF stage

We have already checked the client principal.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
41473daf by Andrew Bartlett at 2018-09-05T09:42:26Z
heimdal: Change KDC to respect HDB server name type if f.canonicalize is set

This changes behaviour flagged as being for Java 1.6.  My hope is that this does not
set f.canonicalize

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
aa07400f by Andrew Bartlett at 2018-09-05T14:17:59Z
kdc: Improve code clarity with extra brackets

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep  5 16:17:59 CEST 2018 on sn-devel-144

- - - - -
e0e86e8a by Andreas Schneider at 2018-09-05T16:22:24Z
smbclient: Fix errors printed if we can connect with SMB1

smbclient -L //server

before:

Reconnecting with SMB1 for workgroup listing.
Connection to earth.milkyway.site failed (Error NT_STATUS_RESOURCE_NAME_NOT_FOUND)
Failed to connect with SMB1 -- no workgroup available

after:

Reconnecting with SMB1 for workgroup listing.
Unable to connect with SMB1 -- no workgroup available

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
eddcbfea by Andreas Schneider at 2018-09-05T16:22:24Z
s3:smbclient: Do not call cli_RNetShareEnum if SMB1 is disabled

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
50f89bcd by Andreas Schneider at 2018-09-05T16:22:24Z
s3:libsmbclient: Do not call cli_RNetShareEnum if SMB1 is disabled

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0dae4e2f by Andreas Schneider at 2018-09-05T16:22:24Z
s3:libsmbclient: Add function to set protocol levels

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
931e64d2 by Andreas Schneider at 2018-09-05T16:22:24Z
s4:torture: Set credentials for libsmbclient correctly

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
778878a3 by Andreas Schneider at 2018-09-05T19:56:11Z
selftest: Run libsmbclient tests with NT1 and SMB3

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Sep  5 21:56:11 CEST 2018 on sn-devel-144

- - - - -
36c3c7d1 by Noel Power at 2018-09-05T21:27:11Z
s4/selftest: enable tdb.python for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ccbecac3 by Noel Power at 2018-09-05T21:27:11Z
python/samba/tests: make password_hash.py py2/py3 compatible

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5bb286ab by Noel Power at 2018-09-05T21:27:11Z
s4/selftest: enable samba.tests.password_hash_gpgme for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1d18ef26 by Noel Power at 2018-09-05T21:27:11Z
python/samba/tests: py3 port for samba.test.password_hash_ldap

user samba.compat.text_type instead of unicode

in py2 text_type is an alias for unicode
in py3 text_type is an alias for str (which does nothing if
the input is already str)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0854cca4 by Noel Power at 2018-09-05T21:27:12Z
s4/selftest: enable samba.tests.password_hash_fl2008 for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b7909eb2 by Noel Power at 2018-09-05T21:27:12Z
s4/selftest: enable samba.tests.password_hash_ldap for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c719b036 by Noel Power at 2018-09-05T21:27:12Z
selftest/knownfail.d: Add PY3 rule for samba.tests.encrypted_secrets

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e96fcb88 by Noel Power at 2018-09-05T21:27:12Z
python/samba/tests: PY3 port of samba.tests.encrypted_secrets

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5ffcc323 by Noel Power at 2018-09-05T21:27:12Z
s4/selftest: enable samba.tests.encrypted_secrets for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8970ec19 by Noel Power at 2018-09-05T21:27:12Z
selftest/knownfail.d: Add PY3 rule for samba.tests.py_credentials

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3fd9b77a by Noel Power at 2018-09-05T21:27:12Z
auth/credentials: py2/py3 credential key needs to return bytes

new_client_authenticator returns a dictionary. The key
'credential' needs to return bytes in Python3, without this
change the the code will attempt to convert the binary data to
a string (resulting sometimes in decode errors).

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c73c9be1 by Noel Power at 2018-09-05T21:27:12Z
python/samba/tests: port samba.tests.py_credentials for py2/py3 compat

+ ord takes 'str' param
+ unicode doesn't exist in py3
+ py3 bytes class doesn't have encode method, try to ensure py2
  code runs unchanged while adapting code (by using get_string)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9dd1271a by Noel Power at 2018-09-05T21:27:12Z
s4/selftest: enable samba.tests.py_credentials for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
94c98200 by Noel Power at 2018-09-05T21:27:13Z
python/samba/tests: fix traffic for py2/py3 compatability

load/save operations return randomly sorted dictionaries in python3
these changes make sure the already sorted expected results are
compared with sorted actual values.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2a97996c by Noel Power at 2018-09-05T21:27:13Z
python/samba/emulate: PY3 port of samba.tests.emulate.traffic_packet

Fixes
+ None cannot be used with '<' or '>' operators
+ ord expects 'str'
+ unicode doesn't exist in py3
+ bytes class does not have encode method

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5c2e06f7 by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: enable samba.tests.emulate.traffic for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6401eba1 by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: samba.tests.emulate.traffic_packet enable for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4ca09e43 by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: samba.tests.blackbox.traffic_replay enabled for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ad172841 by Noel Power at 2018-09-05T21:27:13Z
python/samba/tests: Change native string to binary for py2/py3 compat

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d0553847 by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: Enable samba.tests.blackbox.traffic_learner for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
06f43e9b by Noel Power at 2018-09-05T21:27:13Z
s4/selftest: enable samba.tests.blackbox.traffic_summary for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
42dfc985 by Noel Power at 2018-09-05T21:27:14Z
s4/selftest: Enable samba.tests.blackbox.smbcontrol for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
05862b7b by Noel Power at 2018-09-06T00:27:51Z
s4/selftest: enable samba4.schemaInfo.python for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Sep  6 02:27:51 CEST 2018 on sn-devel-144

- - - - -
bc62182f by Martin Schwenke at 2018-09-06T06:33:59Z
ctdb-tests: Check result of write() in ARP and TCP tests

CTDB -O3 --picky-developer build is failing.  Not sure how this
slipped through.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Thu Sep  6 08:33:59 CEST 2018 on sn-devel-144

- - - - -
c4c6ca5e by Andreas Schneider at 2018-09-06T10:06:05Z
wafsamba: Support clang as C compiler

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Thu Sep  6 12:06:05 CEST 2018 on sn-devel-144

- - - - -
bc188dd7 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: ignore rules and exclude files

Will fix other ones and come back to these later.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
23b3c5b6 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: fix comment indent

The comments in this file will make different PEP8 rules fight.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d02ec9f2 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: fix E101: indentation contains mixed spaces and tabs

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0f67b6e4 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: fix E111: indentation is not a multiple of four

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
24fe8504 by Joe Guo at 2018-09-06T10:10:11Z
PEP8: fix E122: continuation line missing indentation or outdented

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8678fde5 by Joe Guo at 2018-09-06T10:10:12Z
PEP8: fix E123: closing bracket does not match indentation of opening bracket's line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7ff6c198 by Joe Guo at 2018-09-06T10:10:12Z
PEP8: fix E124: closing bracket does not match visual indentation

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dc02a4c3 by Joe Guo at 2018-09-06T10:10:12Z
PEP8: fix E125: continuation line with same indent as next logical line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
deb81991 by Joe Guo at 2018-09-06T10:10:12Z
PEP8: fix E127: continuation line over-indented for visual indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8c2c9794 by Joe Guo at 2018-09-06T13:50:17Z
PEP8: fix E128: continuation line under-indented for visual indent

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Sep  6 15:50:17 CEST 2018 on sn-devel-144

- - - - -
e5afa011 by Noel Power at 2018-09-06T13:51:35Z
python/samba/tests: port samba.tests.dsdb_schema_attributes to PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
7b4db746 by Noel Power at 2018-09-06T13:51:35Z
s4/selftest: enable samba.tests.dsdb_schema_attributes for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
2b4d2810 by Noel Power at 2018-09-06T13:51:35Z
s4/torture/drs: PY3 port for samba4.drs.replica_sync_rodc

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
d18c9dc6 by Noel Power at 2018-09-06T13:51:35Z
s4/selftest: enable samba4.drs.replica_sync_rodc.python for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
59ff17cc by Noel Power at 2018-09-06T13:51:35Z
s3/selftst: enable samba4.tombstone_reanimation for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
2200b22b by Noel Power at 2018-09-06T13:51:36Z
s4/dsdb/tests: port samba4.tombstone_reanimation for PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

- - - - -
4a63ab95 by Noel Power at 2018-09-06T17:24:54Z
s4/selftest/tests: Enabled samba.tests.samba_tool.computer

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett abartlet at samba.org

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Thu Sep  6 19:24:54 CEST 2018 on sn-devel-144

- - - - -
e7612710 by Ralph Boehme at 2018-09-07T09:37:23Z
s3/wscript: fix bison detection

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
193fdbf1 by Ralph Boehme at 2018-09-07T09:37:23Z
s3/wscript: fix flex detection

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
311e1eb6 by Alexander Bokovoy at 2018-09-07T09:37:23Z
wafsamba/samba_autoconf: when setting undefined result, use empty tuple

A difference between waf 1.x and 2.x is that we gained 0 as an undefined
variable in the cache file. This does not allow to differentiate unset
and set to 0 defines.

Force to use empty tuple () to signify unset defines.

Also, fix handling of extra cflags in case of 'strict=True': if
extra_cflags were not defined, we'd append None to the cflags list and
it confuses conf.check() later. 'None' is added to the command line of a
tool executed by the conf.check() which, depending on a tool, may be
treated as an error and cause wrong test result.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
364077d2 by Alexander Bokovoy at 2018-09-07T09:37:23Z
lib/replace/wscript: fix detection of a fallthrough attribute for clang

clang issues a warning but otherwise allows our test to be compiled and
linked. We consider this case a successful pass for a fallthrough
attribute detection while it is an error.

Turn missing declaration warning into an error so that we actually do
not define fallthrough attribute support in case it doesn't really work.

Fixes FreeBSD 11.2 with clang.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4847daf0 by Alexander Bokovoy at 2018-09-07T13:45:36Z
wafsamba/samba_waf18: redefine flex function

There is a bug in waf: flex routine adjusts its inputs against
the task's current working directory but assumes it is being called from
within the build variant directory.

For Samba this means we adjust one level up than the actual work
directory we use to run (bin/ vs bin/default) and flex doesn't find the
source files.

Fix the issue by creating a local override of flex definition that
utilizes the same workd directory for both path adjustment and running
the flex itself.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Sep  7 15:45:36 CEST 2018 on sn-devel-144

- - - - -
d920a725 by Ralph Boehme at 2018-09-07T15:26:14Z
s3:lib/server_contexts: make server_event_ctx and server_msg_ctx static

server_event_ctx and server_msg_ctx static shouldn't be accessible from
outside this compilation unit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
dff1028e by Christof Schmitt at 2018-09-07T15:26:14Z
s3/lib:popt_common: Move setup_logging to common callback

The flag is set in the common callback, so be consistent

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
b7464fd8 by Christof Schmitt at 2018-09-07T15:26:14Z
s3:lib: Move popt_common_credentials to separate file

This is only used by command line utilities and has additional
dependencies. Move to a separate file to contain the dependencies to the
command line tools.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
8c3b62e6 by Christof Schmitt at 2018-09-07T15:26:14Z
s3:lib: Introduce cmdline context wrapper

Command line tools need acccess to the same messaging context provided
by server_messaging_context, as common code for db_open uses that
context. We want to have additional checking for command line tools
without having that code part of the servers. Introduce a wrapper
library to use for command line tools with the additional checks, that
then acquires the server_messaging_context.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
3aca3f24 by Ralph Boehme at 2018-09-07T15:26:14Z
s3:loadparm: reinit_globals in lp_load_with_registry_shares()

This was set to false in 0e0d77519c27038b30fec92d542198e97be767d9 based
on the assumption that callers would have no need to call
lp_load_initial_only() with a later call to lp_load_something().

This is not quite correct, since for accessing registry config on a
cluster with include=registry, we need messaging up and running which
*itself* requires loadparm to be initialized to get the statedir,
lockdir asf. directories.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
10e1a6eb by Ralph Boehme at 2018-09-07T15:26:15Z
selftest: pass configfile to pdbedit

This is needed otherwise pdbedit fails to initialize messaging in
autobuild.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
2c63ce94 by Ralph Boehme at 2018-09-07T15:26:15Z
s3:popt_common: use cmdline_messaging_context() in popt_common_credentials_callback()

This adds a call to cmdline_messaging_context() to the popt
popt_common_credentials_callback() hook and ensures that any client tool
that uses POPT_COMMON_CREDENTIALS gets an implicit messaging context,
ensuring it doesn't crash in the subsequent lp_load_client() with
include=registry in a cluster.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
3ecb9ed7 by Christof Schmitt at 2018-09-07T15:26:15Z
test:doc: Skip 'clustering=yes'

As testparm will error out when running clustering=yes as non-root, skip
this step to avoid a test failure.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
29fd2c2e by Christof Schmitt at 2018-09-07T15:26:15Z
s3:smbpasswd: Use cmdline_messaging_context

smbpasswd does not use POPT_CREDENTIALS. Call cmdline_messaging_context
to initialize a messaging_context with proper error checking before
calling lp_load_global.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
d7fa3815 by Christof Schmitt at 2018-09-07T15:26:15Z
s3:smbstatus: Use cmdline_messaging_context

Use cmdline_messaging_context to initialize a messaging context instead
of open coding the same steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
dd3ae2ff by Christof Schmitt at 2018-09-07T15:26:15Z
rpcclient: Use cmdline_messaging_context

Use cmdline_messaging_context with its error checking instead of open
coding the same steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f2b659e4 by Christof Schmitt at 2018-09-07T15:26:15Z
s3:net: Use cmdline_messaging_context

Use cmdline_messaging_context with its error checking instead of open
coding the same steps.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f56496b1 by Ralph Boehme at 2018-09-07T15:26:15Z
s3:messaging: remove unused messaging_init_client()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
4661537c by Christof Schmitt at 2018-09-07T15:26:16Z
s3:pdbedit: Use cmdline_messaging_context

Initialize the messaging context through cmdline_messaging_context to
allow access to config in clustered Samba.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
ea7a4ff7 by Christof Schmitt at 2018-09-07T15:26:16Z
s3:testparm: Use cmdline_messaging_context

Call cmdline_messaging_context to initialize a messaging config before
accessing clustered Samba config.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
cab8f27b by Christof Schmitt at 2018-09-07T15:26:16Z
s3:sharesec: Use cmdline_messaging_context

Call cmdline_messasging_context to initialize messaging context before
accessing clustered Samba config.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9ed61747 by Christof Schmitt at 2018-09-07T15:26:16Z
s3: ntlm_auth: Use cmdline_messaging_context

Call cmdline_messaging_context to initialize the messaging context
before accessing clustered Samba config.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
095123df by Christof Schmitt at 2018-09-07T15:26:16Z
s3:eventlogadm: Use cmdline_messaging_context

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
7eeff96b by Christof Schmitt at 2018-09-07T15:26:16Z
s3:dbwrap_tool: Use cmdline_messaging_context

Initialize the messaging context through cmdline_messaging_context to
allow access to config in clustered Samba.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
de040eaf by Christof Schmitt at 2018-09-07T15:26:16Z
s3:smbcontrol: Use cmdline_messaging_context

Initialize the messaging context through cmdline_messaging_context to
allow access to config in clustered Samba.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
20ed1392 by Christof Schmitt at 2018-09-07T15:26:17Z
s3:smbget: Use cmdline_messaging_context

Initialize the messaging context through cmdline_messaging_context to
allow access to config in clustered Samba.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
94852e35 by Volker Lendecke at 2018-09-07T15:26:17Z
examples: Fix the smb2mount build

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13465

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
ae7db3e5 by Christof Schmitt at 2018-09-07T15:26:17Z
s3: Rename server_event_context() to global_event_context()

This reflects that the event context is also used outside of the server
processes.

The command used for the rename:
find . -name '*.[hc]' -print0 | xargs -0 sed -i 's/server_event_context/global_event_context/'

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
cc76aaeb by Christof Schmitt at 2018-09-07T15:26:17Z
s3: Rename server_messaging_context() to global_messaging_context()

This reflects that the messaging context is also used outside of the
server processes.

The command used for the rename:
find . -name '*.[hc]' -print0 | xargs -0 sed -i 's/server_messaging_context/global_messaging_context/'

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
a7c19bca by Christof Schmitt at 2018-09-07T15:26:17Z
s3:lib/server_contexts: Rename variables

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
b88290cd by Christof Schmitt at 2018-09-07T15:26:17Z
s3:lib: Rename server_contexts to global_contexts

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
c2281341 by Volker Lendecke at 2018-09-07T15:26:17Z
smbd: Factor out file_has_read_oplocks()

Why? This makes it clearer to me that we're not interested in the actual
number of read oplocks. We only want to know if there are any read
oplocks at all.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
6a782014 by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify logic in smb2_lease_break_send

If/else if chains are hard to follow to me. Simplify the code by using
early returns.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
91085720 by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify logic in fsp_lease_update

One if-condition is better than two

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
45a2e3eb by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify lease_match() a bit

This has been implicitly initialized to 0 with the explicit struct
initializer above.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
6a76a317 by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify share_mode_stale_pid

This loop does not need to count valid share modes. A single valid one
is sufficient for keeping the delete token around

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
6be25d56 by Volker Lendecke at 2018-09-07T15:26:18Z
smbd: Simplify logic in remove_stale_share_mode_entries

To me, an early "continue" is easier to follow than a "else".

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
a5f9b33c by Volker Lendecke at 2018-09-07T15:26:18Z
vfs_fruit: fix an uninitialized variable error

clang does not recognize "smb_panic" as an "exit()" equivalent

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
95682e2f by Volker Lendecke at 2018-09-07T15:26:18Z
dbwrap: Remove a pointless "return;"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
b487979f by Martin Schwenke at 2018-09-07T15:26:18Z
ctdb-tests: Fix CTDB -O3 --picky-developer build on CentOS 7

gcc 4.8.5 complains:

[319/381] Compiling ctdb/tests/src/system_socket_test.c
../tests/src/system_socket_test.c: In function ‘test_tcp’:
../tests/src/system_socket_test.c:196:20: error: ‘rst_out’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
  assert((rst != 0) == (rst_out != 0));
                    ^
cc1: all warnings being treated as errors

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
70169d47 by Andreas Schneider at 2018-09-07T18:24:46Z
wafsamba: Fix 'make -j<jobs>'

Currently only 'make -j' enables parallel builds and e.g. 'make -j4'
results in no parallel compile jobs at all.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13606

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Sep  7 20:24:46 CEST 2018 on sn-devel-144

- - - - -
a2aa7d6d by Günther Deschner at 2018-09-07T23:43:27Z
s3-spoolss: Make spoolss client os_major,os_minor and os_build configurable.

Similar to spoolss server options, make the client advertised OS version
values configurable to allow overriding the defaults provided to the print server.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13597

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
1da77084 by Justin Stephenson at 2018-09-07T23:43:27Z
s3-rpc_client: Advertise Windows 7 client info

Client printing operations currently fail against Windows
Server 2016 with Access Denied if a client os build number
lower than 6000 is advertised. Increase the default build number,
major, and minor versions to values associated with client
OS versoins Windows 7 and Windows Server 2008 R2.

The build number value specifically needs to be increased to
allow these operations to succeed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13597

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b8279f65 by Günther Deschner at 2018-09-07T23:43:27Z
s3-rpcclient: Use spoolss_init_spoolss_UserLevel1 in winspool cmds

Use spoolss initialization function to set client version information for
iremotewinspool printer operations

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
4fc0602d by Justin Stephenson at 2018-09-07T23:43:27Z
iremotewinspool-tests: Allow modification of OS client version information

Add test_get_client_info() function to set and, or modify the client OS
version values advertised in the iremotewinspool torture tests.

The OS build numbers are used from the table in:

  [MS-RPRN] <168> Section 2.2.3.10.1

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
2b970741 by Justin Stephenson at 2018-09-07T23:43:28Z
iremotewinspool-tests: Add client os build number validation test

Add test validating the AsyncOpenPrinter result based on the provided
client info build number

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b1bf9c5d by Justin Stephenson at 2018-09-08T02:35:56Z
spoolss-iremotewinspool-tests: Use more recent client OS version

Set torture test client info build, major, and minor
version numbers to Windows 7 and Windows Server 2008 R2 values

 buildnum: 7007
 major: 6
 minor: 1

Build number taken from
 [MS-RPRN] <168> Section 2.2.3.10.1

Major/Minor numbers taken from
 https://docs.microsoft.com/en-us/windows/desktop/sysinfo/operating-system-version

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Sep  8 04:35:56 CEST 2018 on sn-devel-144

- - - - -
95ba6b97 by Alexander Bokovoy at 2018-09-08T06:44:16Z
wafsamba/samba_autoconf: when setting undefined result, use empty tuple

A difference between waf 1.x and 2.x is that we gained 0 as an undefined
variable in the cache file. This does not allow to differentiate unset
and set to 0 defines.

Force to use empty tuple () to signify unset defines.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8ec7eb0c by Alexander Bokovoy at 2018-09-08T06:44:16Z
lib/replace: force to unset defines if check has failed

For HAVE_WORKING_STRPTIME and HAVE_INCOHERENT_MMAP we always want to
have their defines unset as the code is using '#ifdef' rather than just
'#if'. Setting them to 0 means #ifdef will succeed due to a difference
how '#ifdef' and '#if' are working.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8c00c017 by Jeremy Allison at 2018-09-08T09:37:44Z
s4: torture: Ensure raw notify tests use different directories.

Should help track down flapping tests.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Sep  8 11:37:44 CEST 2018 on sn-devel-144

- - - - -
4e5d6f81 by Douglas Bagnall at 2018-09-10T14:51:09Z
selftext: Mark py3 visualize test as flapping

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Mon Sep 10 16:51:09 CEST 2018 on sn-devel-144

- - - - -
200ddf8e by Stefan Metzmacher at 2018-09-10T17:28:14Z
Revert "lib/replace: force to unset defines if check has failed"

This reverts commit 8ec7eb0c0129518557d1f446191860a62ef3ff79.

There was already a better fix under discusion.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
93b23570 by Stefan Metzmacher at 2018-09-10T17:28:14Z
Revert "wafsamba/samba_autoconf: when setting undefined result, use empty tuple"

This reverts commit 95ba6b97441b75f28aef5ec1ee5a9442683f3763.

There was already a better fix under discusion.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
230796d7 by Alexander Bokovoy at 2018-09-10T17:28:15Z
buildtools/wafsamba: undefine a define for a failed test unless asked to keep it

If conf.CHECK_CODE() is called without `always=True` and the test has
failed, undefine the define already set to '0' by conf.check_code().

This restores expectations that undefined symbols are not considered to
be set by CONFIG_SET() method.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0cd9452e by Stefan Metzmacher at 2018-09-10T17:28:15Z
wafsamba: fix generic_cc.py to work with waf 2

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
420d0b56 by Stefan Metzmacher at 2018-09-10T17:28:15Z
wafsamba: remove irixcc.py as we don't support IRIX any more

There's still a irixcc.py in waf itself.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b35ac32b by Stefan Metzmacher at 2018-09-10T17:28:15Z
wafsamba: remove hpuxcc.py as it's not compatible with waf 2

third_party/waf/waflib/Tools/compiler_c.py proposed
gcc for HPUX.

If there's more needed on HPUX someone with a HPUX box needs to
provide a waf 2 compatible fix.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ad042757 by Stefan Metzmacher at 2018-09-10T20:49:31Z
wafsamba: remove tru64cc.py as it's not compatible with waf 2

third_party/waf/waflib/Tools/compiler_c.py proposed
gcc for osf1V/Tru64.

If there's more needed on Tru64 someone with a Tru64 box needs to
provide a waf 2 compatible fix.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Sep 10 22:49:31 CEST 2018 on sn-devel-144

- - - - -
a122428a by Martin Schwenke at 2018-09-11T04:59:11Z
ctdb-build: Use wafsamba's INSTALL_DIR()

install_dir() doesn't work (doesn't respect $DESTDIR) and is only
available for backward compatibilty.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
26ea0f58 by Martin Schwenke at 2018-09-11T04:59:11Z
wafsamba: Make INSTALL_DIR() respect $DESTDIR

INSTALL_DIR() currently ignores $DESTDIR and just installs directories
into the final destination.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
b4fa3f98 by Martin Schwenke at 2018-09-11T04:59:11Z
wafsamba: Have INSTALL_DIR() log directory creation

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
587bbd3e by Martin Schwenke at 2018-09-11T08:00:05Z
wafsamba: Drop unused, broken install_dir()

This is broken because it doesn't respect $DESTDIR.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Alexander Bokovoy <ab at samba.org>
Autobuild-Date(master): Tue Sep 11 10:00:05 CEST 2018 on sn-devel-144

- - - - -
94c6a7d7 by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Remove an unneeded #include

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dd89034f by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Simplify close_directory()

Same patch as in 8541829a9ab20c7fa8c

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5ef3610f by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Remove an unneeded #include

ndr_open_files already includes open_files.h

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f741f4d1 by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Factor out "has_other_nonposix_opens"

This is exactly the same in both file and directory cases

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e84da7ec by Volker Lendecke at 2018-09-11T16:36:23Z
smbd: Use has_other_nonposix_opens in smb_posix_unlink

Almost the same code as in close.c. has_other_nonposix_opens() is a bit
more general, but the purpose is the same.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e24f3ff4 by Jeremy Allison at 2018-09-11T16:36:23Z
s4: torture: Fix "flakey" notify test on slow cloud filesystems.

Ensure we keep asking for change notifies until we get them all
(or the request times out).

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
885435e8 by Andreas Schneider at 2018-09-11T19:43:28Z
s3:libsmbclient: Fix a typo

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Michael Adam <obnox at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Sep 11 21:43:28 CEST 2018 on sn-devel-144

- - - - -
204e054b by Jeremy Allison at 2018-09-11T20:45:10Z
s3: smbd: Initialization cleanup.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Michael Adam <obnox at samba.org>

- - - - -
de754105 by Jeremy Allison at 2018-09-11T20:45:10Z
s3: smbd: Restructure check_parent_exists() to ensure all non-optimization returns go to one place.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Michael Adam <obnox at samba.org>

- - - - -
ba9d293c by Jeremy Allison at 2018-09-11T20:45:10Z
s3: smbd: Ensure dirpath is set to ".", not "\0" so it is always a valid path.

Cleanup of the internals of unix_convert().

Ensure check_parent_exists() returns this in the non-optimization
case. Ensure unix_convert() initializes dirpath to ".".

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Michael Adam <obnox at samba.org>

- - - - -
0993141c by Michael Adam at 2018-09-12T00:05:47Z
smbd:vfs: fix mis-spellings of hierarchy in comments

Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Sep 12 02:05:47 CEST 2018 on sn-devel-144

- - - - -
3903f6c3 by Martin Schwenke at 2018-09-12T03:50:46Z
ctdb-build: Fix version handling when building tarball

Split get_version() into 2 functions, so that .distversion file can be
created.

Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Wed Sep 12 05:50:46 CEST 2018 on sn-devel-144

- - - - -
cc30805e by Björn Baumbach at 2018-09-12T16:03:10Z
samba_dnsupdate: honor 'dns zone scavenging' option, only update if needed

Since scavenging is implemented the samba_dnsupdate command always updates all
dns records required by the dc. This is not needed if dns zone scavenging
is not enabled.

This avoids the repeating TSIG error messages:

 # samba_dnsupdate --option='dns zone scavenging = yes' 2>&1 | uniq -c
     29 ; TSIG error with server: tsig verify failure
      1 Failed update of 29 entries
 # echo ${PIPESTATUS[0]}
 29

 # samba_dnsupdate --option='dns zone scavenging = no' 2>&1 | uniq -c
 # echo ${PIPESTATUS[0]}
 0

Note that this results in about 60 lines in the log file,
which triggered every 10 minutes ("dnsupdate:name interval=600" is the default).

This restores the behavior before 8ef42d4dab4dfaf5ad225b33f7748914f14dcd8c,
if "dns zone scavenging" is not switched on (which is still the default).

Avoiding the message from happening at all is subject for more debugging,
most likely they are caused by bugs in 'nsupdate -g' (from the bind package).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13605

Pair-programmed-with: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Björn Baumbach <bb at sernet.de>
Signed-off-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Björn Baumbach <bb at sernet.de>
Autobuild-Date(master): Wed Sep 12 18:03:10 CEST 2018 on sn-devel-144

- - - - -
e52abc8a by Ralph Wuerthner at 2018-09-12T19:50:57Z
ctdb-doc: Remove PIDFILE option from ctdbd_wrapper man page

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13610

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Ralph Böhme <slow at samba.org>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Wed Sep 12 21:50:57 CEST 2018 on sn-devel-144

- - - - -
b1838b15 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap: move sockname variable and call to lp_ctdbd_socket into context

sockname is only needed in a cluster.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6617b2db by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap: early return, removes an indentation level

No change in behaviour.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9a9620bb by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_tdb: move a function call out of an if condition

At least for me this improves readability somewhat. No change in
behaviour.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
382620d4 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_tdb: use struct initializer

This ensures all struct members are implicitly initialized.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7cafbc50 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: add error checking to ctdbd_dbpath()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
daea9655 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: simplify if condition

This just moves the talloc_memdup() out of the if condition as per
README.Coding.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dcc50d67 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: README.Coding fixes in traverse_callback()

NULL initialize pointers, check function return values, explicit
variable check against NULL.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fce54bf0 by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: use struct initializer in db_ctdb_traverse_read()

This ensures all struct members are implicitly initialized.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
eaaee00a by Ralph Boehme at 2018-09-12T20:25:19Z
dbwrap_ctdb: use struct initializer in db_ctdb_traverse()

This ensures all struct members are implicitly initialized.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7c0607e9 by Ralph Boehme at 2018-09-12T20:25:20Z
dbwrap_ctdb: increment record count in traverse_callback()

state->count wasn't incremented and is returned at the end of a
dbwrap_traverse().

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f84e372a by Ralph Boehme at 2018-09-12T23:34:30Z
dbwrap_ctdb: return correct record count for a persistent db read-only traverse

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Sep 13 01:34:30 CEST 2018 on sn-devel-144

- - - - -
93d72bcf by Martin Schwenke at 2018-09-13T07:37:23Z
wafsamba: Only use $DESTDIR in INSTALL_DIR() if it is set

Otherwise the leading '/' is stripped and directories are created
relative to the current directory.

This fixes a regression introduced in recent commit
26ea0f58daace4adef7c5bb17f19476083bf3b7b.

Reported-by: Ralph Böhme <slow at samba.org>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Thu Sep 13 09:37:23 CEST 2018 on sn-devel-144

- - - - -
5e4f34e7 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Add some structure protection for durable reconnect

We should consume all data, and the ndr_pull function fills in all
fields. Thus the ZERO_STRUCT(cookie) is not required.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
b6b7c157 by Volker Lendecke at 2018-09-14T05:49:13Z
dbwrap_tdb: Avoid double-call to talloc_get_type_abort

We've already retrieved "ctx" in the callers of db_tdb_fetch_locked_internal().

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
551f8546 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Add a paranoia check for leases

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
e4498de1 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Avoid casts in DEBUG statements

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
41f2d1d1 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Avoid casts in DEBUG statements

Some indentation changed, best viewed with "git show -w"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
aa30fd54 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Remove "file_sync_all" function

Replace with a call to files_forall. Why? I just came across this
function that only has one pretty obscure user. This does not justify
a full library function, IMHO at least.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
92b01adf by Volker Lendecke at 2018-09-14T05:49:13Z
tevent: Fix a docu typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
89e500d8 by Volker Lendecke at 2018-09-14T05:49:13Z
smbd: Fix a false comment

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9e515f7c by Volker Lendecke at 2018-09-14T05:49:14Z
smbd: Simplify share_mode_lock.c

"the_lock_id" is not required here. The share mode data carry the file
id, so use that.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
50adbbe0 by Volker Lendecke at 2018-09-14T05:49:14Z
smbd: Simplify parse_share_modes

Since 823bc4c07ad pidl initializes the [skip] entries

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
36055dbb by Volker Lendecke at 2018-09-14T05:49:14Z
streams_xattr: Make error handling more obvious

Do the NULL check right after the alloc call

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
aca0f2a1 by Volker Lendecke at 2018-09-14T05:49:14Z
streams_xattr: Make error handling more obvious

Do the NULL check right after the alloc call

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
92d3e3a8 by Volker Lendecke at 2018-09-14T05:49:14Z
mdssvc: Simplify sl_unpack_loop

Move a variable declaration closer to its use, avoid a redundant
?true:false;

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
71c70607 by Volker Lendecke at 2018-09-14T05:49:14Z
mdssvc: Move a variable declaration closer to its use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
98cf04c7 by Volker Lendecke at 2018-09-14T05:49:14Z
mdssvc: Move a variable declaration closer to its use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
f8df3f34 by Volker Lendecke at 2018-09-14T08:48:47Z
mdssvc: Move a variable declaration closer to its use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Sep 14 10:48:47 CEST 2018 on sn-devel-144

- - - - -
18ab3ea7 by Christof Schmitt at 2018-09-15T13:18:25Z
torture: Remove unnecesssary #include <fcntl.h>

This is now provided through system/filesys.h.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
181e3966 by Noel Power at 2018-09-15T13:18:25Z
python/samba/provision: PY3 port to run samba.tests.upgradeprovisionneeddc

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5517d526 by Noel Power at 2018-09-15T13:18:25Z
s4/selftest: enable samba.tests.upgradeprovisionneeddc for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
12949bb0 by Noel Power at 2018-09-15T13:18:25Z
s4/selftest: enable samba.tests.samba_tool.rodc for PY3

- - - - -
5e21334d by Noel Power at 2018-09-15T13:18:25Z
s4/dsdb: py_dsdb_DsReplicaAttribute should deal with bytes in py3

Seems the underlying c code expects binary blob, so.. we should
handle str for py2 and byte for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6b30bcf7 by Noel Power at 2018-09-15T13:18:26Z
fallout from py_dsdb_DsReplicaAttribute to -> bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cea2fd6c by Noel Power at 2018-09-15T13:18:26Z
s4/param: py2/p3 compat override_prefixmap should be string/bytes

- - - - -
dcbdae15 by Noel Power at 2018-09-15T13:18:26Z
python/samba: ndr_unpack has to take bytes not string

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fc13a126 by Noel Power at 2018-09-15T13:18:26Z
python/samba: PY3 port for ridalloc_exop test to work

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bf2c6029 by Noel Power at 2018-09-15T13:18:26Z
s4/selftest: enable samba4.drs.ridalloc_exop for python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
939957fc by Noel Power at 2018-09-15T13:18:26Z
python/samba/netcmd: PY3 fix test samba.tests.samba_tool.passwordsettings

- - - - -
7afa2d93 by Noel Power at 2018-09-15T13:18:26Z
s4/selftest: enable samba.tests.samba_tool.passwordsetting for Py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8db43696 by Noel Power at 2018-09-15T13:18:26Z
python/samba: extra ndr_unpack needs bytes function

- - - - -
6941544a by Noel Power at 2018-09-15T13:18:27Z
s4/selftest/tests.py enable samba.tests.getdcname for py3

- - - - -
ed1fd607 by Noel Power at 2018-09-15T13:18:27Z
s4/selftest: enable samba.tests.net_join_no_spnego for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2f5db304 by Noel Power at 2018-09-15T13:18:27Z
s4/selftest: enable samba.tests.net_join for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3530dad6 by Noel Power at 2018-09-15T13:18:27Z
selftest/knownfail.d: add PY3 entry for samba4.ldap.password_settings

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4deb61d6 by Noel Power at 2018-09-15T13:18:27Z
s4/selftest: enable samba4.ldap.password_settings for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1e23962a by Noel Power at 2018-09-15T13:18:27Z
s4/torture/drs: port samba4.drs.samba_tool_drs for py2/py3 compat

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
12ff5df1 by Noel Power at 2018-09-15T13:18:27Z
s4/selftest: enable samba4.drs.samba_tool_drs for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0728cce0 by Noel Power at 2018-09-15T13:18:27Z
s4/torture/drs: PY3 port for test samba_tool_drs_showrepl

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
111fd459 by Noel Power at 2018-09-15T13:18:28Z
s4/torture/drs/python: make test code run in py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c0b18798 by Noel Power at 2018-09-15T13:18:28Z
s4/torture/drs: PY3 fix error with test_samba_tool_showrepl_json

some versions of json.loads appear to need to be passed string.
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0e20bf3d by Noel Power at 2018-09-15T13:18:28Z
selftest/knownfail.d: PY3 rule for replica_sync

- - - - -
13dc820c by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: enable samba4.drs.samba_tool_drs_showrepl py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
89f5f53f by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: samba4.drs.replica_sync.python enable for python3.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
03deaae2 by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: enable samba.tests.samba_tool.forest for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a9295541 by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: enable samba.tests.samba_tool.schema for py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
72497e1f by Noel Power at 2018-09-15T13:18:28Z
s4/torture/drs: port samba4.drs.delete_object test for py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6db0d67f by Noel Power at 2018-09-15T13:18:28Z
s4/selftest: enable test samba4.drs.delete_object for py2/py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
496fc547 by Noel Power at 2018-09-15T13:18:29Z
s4/torture/drs: py2/py3 compat changes for test samba4.drs.fsmo

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a299ff58 by Noel Power at 2018-09-15T13:18:29Z
s4/selftest: enable samba4.drs.fsmo for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4de32576 by Noel Power at 2018-09-15T13:18:29Z
s4/torture/drs: py2/py3 compat for test samba4.drs.repl_move

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
40638fe7 by Noel Power at 2018-09-15T13:18:29Z
s4/selftest: enable samba4.drs.repl_move for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4e6987e5 by Noel Power at 2018-09-15T13:18:29Z
s4/torture/drs/python: ndr_unpack needs bytes samba4.drs.getnc_exop

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
30be125b by Noel Power at 2018-09-15T13:18:29Z
s4/selftest: enable samba4.drs.getnc_exop for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
050ba20f by Noel Power at 2018-09-15T13:18:29Z
s4/selftest: enable samba4.drs.getnc_unpriv for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
71900d92 by Noel Power at 2018-09-15T13:18:29Z
s4/torture/drs: py2/py3 compat porting for samba4.drs.link_conflicts

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9a5a2e06 by Noel Power at 2018-09-15T13:18:30Z
s4/selftest: enable samba4.drs.link_conflicts for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
50c5c47a by Noel Power at 2018-09-15T13:18:30Z
s4/selftest: enable samba4.drs.repl_schema for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
186b7add by Noel Power at 2018-09-15T13:18:30Z
s4/torture/drs/python: py2/py3 porting needed for samba4.drs.getncchanges

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7943e532 by Noel Power at 2018-09-15T13:18:30Z
selftest/knownfail.d: PY3 entry fro getncchanges

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bcefbbb3 by Noel Power at 2018-09-15T13:18:30Z
s4/selftest: enable samba4.drs.getncchanges for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0ed0c07d by Noel Power at 2018-09-15T13:18:30Z
s4/torture/drs/python: py2/py3 compat changes for samba4.drs.repl_rodc

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3451c5e7 by Noel Power at 2018-09-15T13:18:30Z
s4/selftest: enable samba4.drs.repl_rodc for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0fd8c69e by Noel Power at 2018-09-15T13:18:30Z
python/samba/tests: py_gensec_update takes bytes as param

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2eac2f13 by Noel Power at 2018-09-15T13:18:31Z
s4/selftest: enable samba.tests.join for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f18674db by Noel Power at 2018-09-15T13:18:31Z
s4/selftetst: enable samba4.drs.cracknames for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b2665d56 by Noel Power at 2018-09-15T13:18:31Z
s4/selftest: enable samba.tests.kcc for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5884ef29 by Noel Power at 2018-09-15T13:18:31Z
s4/selftest: enable samba.tests.kcc.kcc_utils for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d5413d5b by Noel Power at 2018-09-15T13:18:31Z
s4/selftest: enable samba.tests.netlogonsvc for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7208d4a4 by Noel Power at 2018-09-15T13:18:31Z
selftest/knownfail: Add PY3 entries for samba.tests.ntlmdisabled

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4038e7bd by Noel Power at 2018-09-15T16:10:42Z
s4/selftest: enable samba.tests.ntlmdisabled for py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Sep 15 18:10:42 CEST 2018 on sn-devel-144

- - - - -
3cc284b2 by Noel Power at 2018-09-16T04:16:19Z
PY3: fix some octal literals

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
693834f7 by Noel Power at 2018-09-16T04:16:19Z
PY3: fix iteritems usage, not supported in python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c6884138 by Noel Power at 2018-09-16T04:16:19Z
PY3: dict has no iterkeys method

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d20e85d0 by Noel Power at 2018-09-16T04:16:19Z
PY3: iterkeys -> keys

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b2b0f819 by Noel Power at 2018-09-16T04:16:19Z
PY3: iterkeys -> keys

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
82c84513 by Noel Power at 2018-09-16T04:16:19Z
PY3: dict doesn't have has_key

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0358d57d by Noel Power at 2018-09-16T04:16:19Z
PY3: dict_keys doesn't have a sort method

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
59f69a7c by Noel Power at 2018-09-16T04:16:19Z
py2/py3 enclose map with list

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bb9e945e by Noel Power at 2018-09-16T04:16:19Z
PY3: fix string.rsplit usage (doens't exist in PY3)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9fc67597 by Noel Power at 2018-09-16T04:16:19Z
PY3: enclose filter with list as len on result of filter fails

filter returns an iterator in PY3 (and a list in PY2)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b342e6f0 by Noel Power at 2018-09-16T04:16:19Z
PY3: bytes.maketrans, string.maketrans is a PY2 only function

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a0c8c8c8 by Noel Power at 2018-09-16T04:16:19Z
PY3: string.translate string.upper don't exist in PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
eceb9970 by Noel Power at 2018-09-16T04:16:20Z
PY3: decode output of cmd_output for easier string manip

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
899ef5d1 by Noel Power at 2018-09-16T04:16:20Z
PY3: string.upper not in PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e2c0af6b by Noel Power at 2018-09-16T04:16:20Z
PY3: xrange->range

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0df8ef41 by Noel Power at 2018-09-16T04:16:20Z
PY3: use pyembed_libname for library names (otherwise missing deps)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6c62e22c by Noel Power at 2018-09-16T04:16:20Z
PY3: detect talloc_utils (for version of python running the build)

It seems all combinations have potential to fail here :/ however for
the moment making the assumption
  *  if python2 driving the build you are building against libpython2 (with the option of having python3 as 'extra-python'
  *  if python3 driving the build you are building against libpython3 (with the option of having python3 as 'extra-python' NOTE: this isn't inforced it probably should be !!!! In a system with python2 and python3 installed it seems waf needs PYTHON env variable to decide which libpython to build against. It's also an option that configure should use that too (to figure out which talloc_util to use or build)

However.... right now I just want the thing to build

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dc9117e4 by Noel Power at 2018-09-16T04:16:20Z
PY3: ensure output of subprocess.Popen is decoded

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a0851c5f by Noel Power at 2018-09-16T04:16:20Z
PY3: wrap filter calls with list where list is expected

filter in PY2 returns list in PY3 it returns an iterator

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0cb34132 by Noel Power at 2018-09-16T07:21:59Z
s4/librpc: Fix py2 dependecies leaking into py3 libraries

Caused strange crashes when using python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sun Sep 16 09:21:59 CEST 2018 on sn-devel-144

- - - - -
5a6b1398 by Martin Schwenke at 2018-09-17T20:58:19Z
ctdb-cluster-mutex: Reset SIGTERM handler in cluster mutex child

If SIGTERM is received and the tevent signal handler setup in the
recovery daemon is still enabled then the signal is handled and a
corresponding event is queued.  The child never runs an event loop so
the signal is effectively ignored.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e789d0da by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-cluster-mutex: Block signals around fork

If SIGTERM is received and the tevent signal handler setup in the
recovery daemon is still enabled then the signal is handled and a
corresponding event is queued.  The child never runs an event loop so
the signal is effectively ignored.

Resetting the SIGTERM handler isn't enough.  A signal can arrive
before that.

Block SIGTERM before forking and then immediately unblock it in the
parent.

In the child, unblock SIGTERM after the signal handler is reset.  An
explicit unblock is needed because according to sigprocmask(2) "the
signal mask is preserved across execve(2)".

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
59fc0164 by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Clean up taking of recovery lock

No functional changes, just coding style cleanups and debug message
tweaks.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c516e58c by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Re-check master on failure to take recovery lock

If the master changed while trying to take the lock then fail gracefully.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
af22f03d by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Rename hold_reclock_state to ctdb_recovery_lock_handle

This will be a longer lived structure.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a53b264a by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Use talloc() to allocate recovery lock handle

At the moment this is still local and is freed after the mutex is
successfully taken.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c5221674 by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Store recovery lock handle

... not just cluster mutex handle.

This makes the recovery lock handle long-lived and with allow the
releasing code to cancel an in-progress attempt to take the recovery
lock.

The cluster mutex handle is now allocated off the recovery lock
handle.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a755d060 by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Return early when the recovery lock is not held

This makes upcoming changes simpler.

Update to modern debug macro while touching relevant line.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
b1dc5687 by Martin Schwenke at 2018-09-17T20:58:20Z
ctdb-recoverd: Handle cancellation when releasing recovery lock

If the recovery lock is in the process of being taken then free the
cluster mutex handle but leave the recovery lock handle in place.
This allows ctdb_recovery_lock() to fail.

Note that this isn't yet live because rec->recovery_lock_handle is
still only set at the completion of the attempt to take the lock.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
486022ef by Martin Schwenke at 2018-09-18T00:18:30Z
ctdb-recoverd: Set recovery lock handle at start of attempt

This allows the attempt to be cancelled if an election is lost and an
unlock is done before the attempt is completed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13617

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Tue Sep 18 02:18:30 CEST 2018 on sn-devel-144

- - - - -
d0d1eafa by Martin Schwenke at 2018-09-18T08:33:57Z
gitlab-ci: Correct the ctdb tasks

They're swapped.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Tue Sep 18 10:33:57 CEST 2018 on sn-devel-144

- - - - -
8453c617 by Christof Schmitt at 2018-09-18T22:54:23Z
vfs_gpfs: Check for GPFS file system on connect

The vfs_gpfs modules uses GPFS API calls that only succeed when using
the module with the GPFS file system. Add an explicit statfs check for
the file system type on connect, to make it obvious when the file system
is missing or not mounted. The check can be skipped by setting
gpfs:check_fstype to 'no'.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
8b4ec1c2 by Christof Schmitt at 2018-09-19T01:56:47Z
docs: Add gpfs:check_fstype to vfs_gpfs manpage

Signed-off-by; Christof Schmit <cs at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Wed Sep 19 03:56:47 CEST 2018 on sn-devel-144

- - - - -
e5298c8b by Noel Power at 2018-09-19T15:49:33Z
uid_wrapper: Fix Tab/indent error with python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Wed Sep 19 17:49:33 CEST 2018 on sn-devel-144

- - - - -
48ad1db3 by Richard Sharpe at 2018-09-19T15:52:13Z
s3: Don't copy back the stat struct from stat_cache_lookup if we did not get a hit on the full path.

Signed-off-by: Richard Sharpe <realrichardsharpe at gmail.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c3211c8e by Anoop C S at 2018-09-19T20:23:28Z
s3/locking: Fix comments to reflect code flow

* posix_pending_close_db is an in-memory database and not a tdb.
  Therefore adjusting comments to convey the correct meaning of the
  database.
* we do not have posix_locking_close_file() any more which got renamed
  to locking_close_file(). Thus fixing comment to mention the new name.

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Sep 19 22:23:28 CEST 2018 on sn-devel-144

- - - - -
2e59a334 by Noel Power at 2018-09-19T20:25:05Z
PY3: make sure print stmt is enclosed by '(' & ')'

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1d1cd28a by Andrew Bartlett at 2018-09-19T20:25:05Z
s3: nmbd: Stop nmbd network announce storm.

Correct fix for. On announce, work->lastannounce_time is set
to current time t, so we must check that 't >= work->lastannounce_time',
not 't > work->lastannounce_time' otherwise we end up not
doing the comparison, and always doing the announce.

Reported by Reuben Farrelly

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13620

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Revviewe-by: Andreas Schneider <asn at samba.org>

- - - - -
cf7e7710 by Andrew Bartlett at 2018-09-19T23:29:47Z
Fix pdb_samba_dsdb build on Ubuntu 16.04 with -O3 -Werror=strict-overflow

../source3/passdb/pdb_samba_dsdb.c: In function ‘pdb_samba_dsdb_set_trusteddom_pw’:
../source3/passdb/pdb_samba_dsdb.c:2778:6: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow]
   if (i == 0) {
      ^
cc1: all warnings being treated as errors

gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Sep 20 01:29:48 CEST 2018 on sn-devel-144

- - - - -
b6efa150 by Amitay Isaacs at 2018-09-21T01:06:44Z
heimdal-build: Avoid using python str.format()

In python 2.6, the format fields need to be numbered explicitly.  This
causes the build to fail on RHEL6/Centos6 with following error:

 File "/home/amitay/samba.git/source4/heimdal_build/wscript_build", line 87, in HEIMDAL_ASN1
    os.path.join(bld.path.abspath(), option_file)))
ValueError: zero length field name in format

To use str.format() in HEIMDAL_ASN1(),

   "--option-file='{}'"  needs to be   "--options-file='{0}'"

Or this fix which avoids str.format() completely.

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Fri Sep 21 03:06:44 CEST 2018 on sn-devel-144

- - - - -
3b94d31d by Amitay Isaacs at 2018-09-21T18:04:22Z
ldb: Fix dependency of ldb module

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13626
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
198bcfba by Douglas Bagnall at 2018-09-21T18:04:23Z
python tests: fix format() strings for Python 2.6

Python 2.6 wants "{0}".format(x), not "{}".format(x).

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
93bbe2d2 by Tim Beale at 2018-09-21T18:04:23Z
tests: Minor code cleanups to confidential_attr test

+ fix a couple of flake8 warnings
+ add some extra code comments (particularly around the cases where the
  child class overrides a particular method, to avoid confusion when
  browsing the code).
+ assert_not_in_result() was duplicated (it's only needed for the deny
  ACL tests)
+ skip redundant if in dirsync's assert_search_result() (it always has
  to use the base-DN - we never pass it this as an args).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6c1ff590 by Tim Beale at 2018-09-21T18:04:23Z
acl_read: Rework Samba code to reflect Windows logic

This patch should not alter functionality. It is just updating the Samba
code to better match the Windows specification docs.

When fixing Samba BUG #13434, the Microsoft behaviour wasn't clearly
documented, so we made a best guess based on observed behaviour.
The problem was an exception was made to allow "objectClass=*" searches
to return objects, even if you didn't have Read Property rights for the
object's objectClass attribute. However, the logic behind what
attributes were and weren't covered by this exception wasn't clear.

I made a guess that it was attributes belonging to the Public Info
property-set that also have the systemOnly flag set.

Microsoft have confirmed the object visibility behaviour. It turns out
that an optimization is made for the 4 attributes that are always
present for every object (i.e. objectClass, distinguishedName,
name, objectGUID). They're updating their Docs to reflect this.

Now that we know the Windows logic, we can update the Samba code.
This simplifies the code somewhat.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5d866751 by Tim Beale at 2018-09-21T18:04:23Z
tests: test ldap search requesting non-existent attribute

As an after-thought to commit 563e454e8c55e94a950, we thought it
might be a good idea to add a test case that requests an non-existent
attribute in the attribute-filter as well the search-filter.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
18854107 by Tim Beale at 2018-09-21T20:58:17Z
selftest: Tweak PSO test-suite name

There are 2 different PSO tests:
- make test TESTS=ldap.password_settings
- make test TESTS=samba_tool.passwordsettings

There's also another test that's completely unrelated to PSOs:
- make test TESTS=blackbox.password_settings

This patch renames ldap.password_settings --> ldap.passwordsettings.
This means 'make test TESTS=passwordsettings' will run both PSO tests,
but not the unrelated blackbox test.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Sep 21 22:58:17 CEST 2018 on sn-devel-144

- - - - -
d50aafe9 by Philipp Gesang at 2018-09-22T04:05:06Z
lib/audit_logging: make json_{is_invalid,to_string}() accept a const*

Allow for json_is_invalid() and json_to_string() to be used on a
const pointer. Neither function requires for the json object to
be mutable so constraining them to non-const* is unnecessary.

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2e00ad44 by Philipp Gesang at 2018-09-22T04:05:06Z
s3: net: implement json output for ads info

Add the switch '--json' to 'net' to format the output as JSON.

The rationale is to supply the information in a machine-readable
fashion to complement the text version of the output which is
neither particularly well defined nor locale-safe.

The output differs from that of plain 'info' in that times are
not formatted as timestamps.

Currently affects only the 'net ads info' subcommand.

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5534b924 by Philipp Gesang at 2018-09-22T04:05:06Z
s3: net: implement json output for ads lookup

Add JSON printer (option '--json') for the 'net ads lookup'
command. This outputs the same information as the plain version,
with integral ({LMNT,LM20} Token, NT Version) and boolean values
(Flags) not stringified.

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2a19b4e3 by Philipp Gesang at 2018-09-22T04:05:07Z
s3: net: normalize output of lookup subcommand

Use spaces and tabs consistently following the majority of the
printed output: tabs only for indenting, no space before the
colon separator, a single space after the separator.

The irregularities in formatting date back to the original commit
2c029a8b96..

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6dc9f70b by Philipp Gesang at 2018-09-22T07:20:09Z
tests/blackbox: add test for net ads JSON output

Implement blackbox tests for

    $ net ads info --json
    $ net ads lookup --json

that validate

    a) JSON wellformedness (by feeding it into the JSON library
       that ships with Python), and
    b) equality of the set of keys printed to that of the
       non-JSON version.

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Sep 22 09:20:09 CEST 2018 on sn-devel-144

- - - - -
08a5fbd8 by Martin Schwenke at 2018-09-24T05:03:09Z
s3: Fix the build when compiling without JSON support

[3762/3895] Compiling source3/utils/net_ads_gpo.c
../source3/utils/net_ads.c: In function ‘net_ads_cldap_netlogon_json’:
../source3/utils/net_ads.c:311:2: error: parameter name omitted
  (ADS_STRUCT *, const char *,
  ^
../source3/utils/net_ads.c:311:2: error: parameter name omitted
../source3/utils/net_ads.c:312:16: error: parameter name omitted
   const struct NETLOGON_SAM_LOGON_RESPONSE_EX *)
                ^
../source3/utils/net_ads.c: In function ‘net_ads_info_json’:
../source3/utils/net_ads.c:520:1: error: parameter name omitted
 static int net_ads_info_json(ADS_STRUCT *)
 ^

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Mon Sep 24 07:03:09 CEST 2018 on sn-devel-144

- - - - -
2055b710 by Tim Beale at 2018-09-26T05:49:17Z
netcmd: Tweak backup-offline output to avoid subunit truncation

Currently a backup-offline test is occasionally flapping in autobuild,
however, the output is truncated so we can't see what the actual problem
is. The output only ever contains the list of backup dirs. I suspect
that the ']' character printed at the end of the python list might be
getting interpretted by subunit as the end of *all* the output.

If so, we should be able to avoid the problem by printing the list items
without the '['/']'s, i.e. join the list into a single string.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e1f255a4 by Tim Beale at 2018-09-26T05:49:17Z
netcmd: Add --site option when restoring a domain

Restoring a backup only worked if the Default-First-Site-Name site was
still present. When the new restored DC account is created, it was
trying to add the new server's DN under CN=Default-First-Site-Name.
However, if the original domain was setup using a different site, then
the restore would fail because the DN didn't exist.

When running the restore command, you should be able to specify the
site that you want the new/restored DC to be in (same as during a
DC 'join'). Passing the correct --site argument is one way to avoid
this problem. (A subsequent patch will further improve the tool so it
can work around non-default sites automatically).

Note we also need to pass the site through to where the new DNS entries
get registered (in the rename case).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ad69aaf7 by Tim Beale at 2018-09-26T05:49:17Z
tests: Add test-case for restore into non-default site

Add a test-case that exercises the new '--site' restore option and
ensures the restored DC gets added to the correct site.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ce57a800 by Tim Beale at 2018-09-26T05:49:17Z
netcmd: Re-create default site for backup-restore (if missing)

Normally when a new DC joins a domain, samba-tool works out the new
DC's site automatically. However, it does this by querying the existing
DC using CLDAP. In the restore case, there is no DC running. We could
still query the DB on disk and work out the correct site based on the
new DC's IP, however:
- comparing between the CN=Subnet DNs and an IP-address string seems
  like it'd be non-trivial to write, and
- in the lab-domain rename case, chances are the user will want a
  completely different subnet to what's already in the DB.

The restore command now has a --site option so the user can specify an
appropriate site for the restored DC. This patch makes the restore
command work by default (i.e. without a --site option) even if the
default Default-First-Site-Name doesn't exist. Basically the solution is
to just check Default-First-Site-Name exists and create it if it
doesn't. As the recommended workflow is to use the restored DC as a
temporary seed that you'll later throw away, this approach seems
acceptable. Subsequent DCs will then be joined to the running restored
DC, so an appropriate site will be determined using CLDAP. The only
side-effect is potentially an extra Site object.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
696fa6a1 by Tim Beale at 2018-09-26T05:49:17Z
selftest: Change backup testenvs to use non-default site

Previously (i.e. up until the last patch) the backup/restore commands
only worked if the Default-First-Site-Name site was present. If this
site didn't exist, then the various restore testenvs would fail to
start. This is now fixed, but this patch changes the backupfrom testenv
so that it uses a non-default site. This will detect the problem if it
is ever re-introduced.

To do this we need to change provision_ad_dc() so the
extra_provision_options can be specified as an argument. (Note that Perl
treats undef the same as an empty array).

By default, the restore will add the new DC into the
Default-First-Site-Name site. This means the backupfromdc and restored
testenvs will now have different sites, so we need to update the ldapcmp
filters to exclude site-specific attributes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
89543af0 by Tim Beale at 2018-09-26T05:49:17Z
selftest: Test join against DC with non-default site

Recent changes around restoring a domain that lacked
Default-First-Site-Name highlighted a problem. Normally when you join a
DC to a domain, samba-tool works out the correct site to use
automatically. However, if the join uses '--server' to select a DC, then
this doesn't work. It defaults back to Default-First-Site-Name, and the
join command fails if this site doesn't exist.

All the testenvs had Default-First-Site-Name present, so this was never
tested. Now the backupfromdc no longer has a Default-First-Site-Name
site, so running a simple join against that DC fails, highlighting the
problem.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9e81d43a by Tim Beale at 2018-09-26T05:49:17Z
join: Avoid duplicating "Default-First-Site-Name" string

The provision code already defines "Default-First-Site-Name" so we might
as well reuse it.

The join.py already uses a suitable default, so assigning the default in
the domain netcmd code is unnecessary.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
26dd30d6 by Tim Beale at 2018-09-26T05:49:18Z
join: Support site detection when --server is specified

When a new DC is joined to the domain, samba-tool would automatically
detect an appropriate site for the new DC. However, it only did this if
the --server option wasn't specified. The new DC's site got
automatically updated as part of the finddc() work, however, this step
gets skipped if we already know the server DC to join to.

In other words, if Default-First-Site-Name doesn't exist and you specify
--server in the join, then you have to also specify --site manually,
otherwise the command fails. This is precisely what's happening in the
join_ldapcmp.sh test, now that the backupfromdc testenv no longer has the
Default-First-Site-Name present.

This patch adds a new find_dc_site() function which uses the same
net.finddc() API (except based on the server-address rather than
domain-name). Assigning DEFAULTSITE has been moved so that it only
gets done if finddc() can't determine the site.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4e592c91 by Douglas Bagnall at 2018-09-26T05:49:18Z
domain_backup test: fix py2.6 incompatible format

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
36cb85be by Douglas Bagnall at 2018-09-26T08:40:49Z
samba-tool domain backup: fix py2.6 incompatible format

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep 26 10:40:49 CEST 2018 on sn-devel-144

- - - - -
cb8e61b5 by Andreas Schneider at 2018-09-26T18:10:07Z
docs: Only build vfs manpages if the module is enabled

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

Autobuild-User(master): Günther Deschner <gd at samba.org>
Autobuild-Date(master): Wed Sep 26 20:10:07 CEST 2018 on sn-devel-144

- - - - -
c98f9971 by Philipp Gesang at 2018-09-26T18:41:07Z
turn --with-json-audit into global --with-json

Fold the build option --with-json-audit into the toplevel wscript
to reflect the fact that JSON support is no longer local to the
audit subsystem.

Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5a73f904 by Douglas Bagnall at 2018-09-26T21:40:58Z
py3 kcc tests: mark verify test as flapping

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Sep 26 23:40:58 CEST 2018 on sn-devel-144

- - - - -
d786e1fc by Noel Power at 2018-09-26T23:54:26Z
lib/ldb: Test correct variable for no mem condition

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d1492ab9 by Noel Power at 2018-09-26T23:54:26Z
lib/ldb/tests: add test for ldb.Dn passed utf8 unicode

object dn format should be a utf8 encoded string
Note: Currently this fails in python2 as the c python binding for
      the dn string param uses PyArg_ParseTupleAndKeywords() with 's'
      format, this will accept str *or* unicode in the default encoding.
      The default encoding in python2 is... ascii.

Also adding here a knownfail to squash the error produced by the test.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cddd54e8 by Noel Power at 2018-09-26T23:54:26Z
lib/ldb: Ensure ldb.Dn can accept utf8 encoded unicode

Additionally remove the associated known fail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13616
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e8fec948 by Noel Power at 2018-09-26T23:54:27Z
PY3: fix "TabError: inconsistent use of tabs and spaces"

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2d94839f by Noel Power at 2018-09-26T23:54:27Z
PY3: decode bytes in py3 where strings are needed

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d28b4fa3 by Noel Power at 2018-09-26T23:54:27Z
PY3: md5 related functions need to be passed bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f71675dc by Noel Power at 2018-09-26T23:54:27Z
python/samba: PY3 add compat function urllib_join to replace urllib.urljoin

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2ea47f3e by Noel Power at 2018-09-26T23:54:27Z
s4/web_server: PY3: port to python3

Note: Unlike other libraries this library is been only built for the
configured python version. It depends on availability of 'swat' python
module.

The swat module is hosted externally, it seems not to have been modified
for a number of years, I don't think swat is python3 compatabile. These
changes are enough to get allow the samba binary to launch a web server
(which will just display a placeholder page announcing you need to install
swat). It maybe that removing this functionality is what we should do, but
that is a decision that can be made at a later time.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9a6052b5 by Noel Power at 2018-09-26T23:54:27Z
buildtools/wafsamba: fix basestring not defined error in PY3

Test for str first (which exists in py3 & py2) this avoids
the undefined runtime error.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8514b5b1 by Noel Power at 2018-09-26T23:54:27Z
buildtools/wafsamba: add_manual_dependency needs bytes for value

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1c16c0cf by Aaron Haslett at 2018-09-26T23:54:27Z
netcmd: domain backup offline bug fix - ignore sock files

Ignoring autogenerated .sock files and directories during file system based
offline domain backup.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13604
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8c754b43 by Aaron Haslett at 2018-09-27T02:47:16Z
dlz_bind9: torture tests exploring rndc reload behaviour

These tests establish that the process triggered by the command 'rndc reload'
does not cause samba's bind9 dlz plugin to crash.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13214

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Sep 27 04:47:16 CEST 2018 on sn-devel-144

- - - - -
6400c8c6 by Noel Power at 2018-09-27T23:01:23Z
buildtools/wafsamba: Finally fix reference to basestring PY3 error.

While a previous attempt squashed the error on the config & make
phase, make install threw up this error again.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
47634a25 by Noel Power at 2018-09-27T23:01:23Z
PY3: We support python3 now, remove error when python3 detected

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
de1e6d78 by Noel Power at 2018-09-27T23:01:23Z
script: add new autobuild task for building pure python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f08ba3a4 by Noel Power at 2018-09-27T23:01:23Z
add new gitlab CI job for building pure python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9c0a9c69 by Noel Power at 2018-09-28T01:57:47Z
s4/web_server: fix panic from wrong type extracted from conn

enabling
  server services = +web

and we get a panic

web_server.c:251: Type mismatch: name[struct web_server_data] expected[struct task_server]
smb_panic_default: PANIC (pid 29137): ../source4/web_server/web_server.c:251: Type mismatch: name[struct web_server_data] expected[struct task_server]

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Sep 28 03:57:47 CEST 2018 on sn-devel-144

- - - - -
0122f45f by Tim Beale at 2018-09-28T06:30:22Z
netcmd: Make sure SMB connection is signed when backing up sysvol

i.e. protect the client against man-in-the-middle attacks by default.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
37870626 by Tim Beale at 2018-09-28T06:30:22Z
s3/smbd: Server responds incorrectly if no SMB protocol chosen

The SMBnegprot response from the server contains the DialectIndex of the
selected protocol from the client's request message. Currently, if no
protocol is selected, the server is responding with a DialectIndex=zero,
which is a valid index (PROTOCOL_CORE by default). The Windows spec, and
historically the code, should return DialectIndex=0xffff if no protocol
is chosen. The following commit changed it recently (presumably
inadvertently), so that it now returns DialectIndex=zero.

06940155f315529c5b5 s3:smbd: Fix size types in reply_negprot()

This results in somewhat confusing error messages on the client side:
ERROR(runtime): uncaught exception - (3221225997, 'The transport
connection has been reset.')

or, when signing is configured as mandatory:
smbXcli_negprot: SMB signing is mandatory and the selected protocol
level (1) doesn't support it.
ERROR(runtime): uncaught exception - (3221225506, '{Access Denied} A
process has requested access to an object but has not been granted those
access rights.')

This patch restores the old behaviour of returning 0xffff.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Pair-Programmed-With: Ralph Boehme <slow at samba.org>
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
34cbd89f by Tim Beale at 2018-09-28T09:25:29Z
libcli: Add debug message if fail to negoatiate SMB protocol

Currently if the client and server can't negotiate an SMB protocol, you
just get the followiing error on the client-side, which doesn't tell you
much.
ERROR(runtime): uncaught exception - (3221225667, 'The network responded
incorrectly.')

This patch adds a debug message to help highlight what's actually going
wrong.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Sep 28 11:25:29 CEST 2018 on sn-devel-144

- - - - -
660dbfae by Jeremy Allison at 2018-09-29T03:32:41Z
s3: smbd: Prevent valgrind errors in smbtorture3 POSIX test.

Missing fsp talloc free and linked list delete in error
paths in close_directory(). Now matches close_normal_file()
and close_fake_file().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13633

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Sep 29 05:32:41 CEST 2018 on sn-devel-144

- - - - -
46e171e2 by Volker Lendecke at 2018-10-02T08:40:05Z
libsmb: Remove smb_share_modes.[ch]

This was declared nonfunctional in 2014. Finally remove it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
380066d2 by Anoop C S at 2018-10-02T08:40:05Z
s3/locking: Fix logging of lock reference count

lock reference count is always increased and reduced by a value of 1.
But lock_ref_count variable holds the old value prior to change and
was being logged wrongly under debug level 10. DEBUG statement must
log lock_ref_count+1 and lock_ref_count-1 respectively when value
gets increased and decreased.

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
01b86845 by Stefan Metzmacher at 2018-10-02T08:40:06Z
s4:torture: split smb2.session.expire{1,2} to run with signing and encryptpion

This reproduces the problem we have with expired encrypted sessions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13624

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4ef45e53 by Stefan Metzmacher at 2018-10-02T12:11:30Z
smb2_server: set req->do_encryption = true earlier

The STATUS_SESSION_EXPIRED error was returned unencrypted,
if the request was encrypted.

If clients use SMB3 encryption and the kerberos authenticated session
expires, clients disconnect the connection instead of doing a reauthentication.

>From https://blogs.msdn.microsoft.com/openspecification/2012/10/05/encryption-in-smb-3-0-a-protocol-perspective/

  The sender encrypts the message if any of the following conditions is
  satisfied:

    - If the sender is sending a response to an encrypted request.
    - If Session.EncryptData is TRUE and the request or response being
      sent is not NEGOTIATE.
    - If Session.EncryptData is FALSE, the request or response being sent
      is not NEGOTIATE or SESSION_SETUP or TREE_CONNECT, and
      <TreeConnect|Share>.EncryptData is TRUE.

[MS-SMB2] 3.3.4.1.4 Encrypting the Message

 If Connection.Dialect belongs to the SMB 3.x dialect family and
 Connection.ClientCapabilities includes the SMB2_GLOBAL_CAP_ENCRYPTION
 bit, the server MUST encrypt the message before sending, if any of the
 following conditions are satisfied:

 - If the message being sent is any response to a client request for which
   Request.IsEncrypted is TRUE.

 - If Session.EncryptData is TRUE and the response being sent is not
   SMB2_NEGOTIATE or SMB2 SESSION_SETUP.

 - If Session.EncryptData is FALSE, the response being sent is not
   SMB2_NEGOTIATE or SMB2 SESSION_SETUP or SMB2 TREE_CONNECT, and
   Share.EncryptData for the share associated with the TreeId in the SMB2
   header of the response is TRUE.

 The server MUST encrypt the message as specified in section 3.1.4.3,
 before sending it to the client.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13624

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Oct  2 14:11:30 CEST 2018 on sn-devel-144

- - - - -
20302506 by Stefan Metzmacher at 2018-10-02T12:12:13Z
python/tests: turn GraphError into failure inside of test_verify()

That test was already marked as flapping in commit
5a73f904e192c44e304850287ac439d0b52f7be5.

However the test generates an UNEXPECTED(error)
instead of an UNEXPECTED(failure).

  [67(594)/75 at 36m28s] samba.tests.kcc.python3(vampire_dc)
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  repsFrom source DSA guid (62f01940-a784-4e60-947a-a661c787c8cc) not found
  UNEXPECTED(error): samba.tests.kcc.python3.samba.tests.kcc.KCCTests.test_verify(vampire_dc)
  REASON: Exception: Exception: Traceback (most recent call last):
    File "bin/python/samba/tests/kcc/__init__.py", line 80, in test_verify
      attempt_live_connections=False)
    File "bin/python/samba/kcc/__init__.py", line 2659, in run
      ('connected',))
    File "bin/python/samba/kcc/__init__.py", line 2513, in plot_all_connections
      vertex_colors=vertex_colours)
    File "bin/python/samba/kcc/graph_utils.py", line 334, in verify_and_dot
      for p, e, doc in errors)))
  samba.kcc.graph_utils.GraphError: The 'dsa_final CN=NTDS Settings,CN=LOCALVAMPIREDC,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=samba,DC=example,DC=com' graph lacks the following properties:
  connected: the graph is not connected, as the following vertices are unreachable:

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
1fd2a79a by Andreas Schneider at 2018-10-02T12:12:13Z
third_party: Update pam_wrapper to version 1.0.7

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
7dd388a1 by Mathieu Parent at 2018-10-02T12:12:13Z
third_party: Add pam_set_items.so from pam_wrapper

Pair-Programmed-With: Andreas Schneider <asn at samba.org>
Signed-off-by: Mathieu Parent <math.parent at gmail.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
ad5debcb by Mathieu Parent at 2018-10-02T12:12:13Z
nsswitch: Add try_authtok option to pam_winbind

Same as the use_authtok option, except that if the new password is not
valid, PAM will prompt for a password.

Bug-Debian: https://bugs.debian.org/858923
Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/570944

Signed-off-by: Mathieu Parent <math.parent at gmail.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

- - - - -
62400224 by Mathieu Parent at 2018-10-02T15:30:29Z
tests: Check pam_winbind pw change with different options

Pair-Programmed-With: Andreas Schneider <asn at samba.org>

Signed-off-by: Mathieu Parent <math.parent at gmail.com>
Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Tue Oct  2 17:30:29 CEST 2018 on sn-devel-144

- - - - -
0e7c5464 by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Move downgrade_share_lease() to smbd/oplock.c

This function is pretty closely entangled with its only caller. In
particular the NT_STATUS_OPLOCK_BREAK_IN_PROGRESS triggers acitivity
in the caller, and that's the only case where "*_l" is being set to
non-NULL. Prepare for cleanup

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
4980e60d by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Use find_share_mode_lease() in downgrade_share_lease

Simple simplification: In locking/ we did not have the direct
reference to find_share_mode_lock.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
664808af by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Slightly simplify downgrade_lease()

As much as I dislike }else{ and prefer early returns, I even more
dislike asking for the same condition in two different ways.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
56139b8e by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Simplify downgrade_lease

To me, the "additive" SMB2_LEASE_WRITE|SMB2_LEASE_HANDLE is easier to
read than the negated ~SMB2_LEASE_READ.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
a93aa151 by Volker Lendecke at 2018-10-02T16:13:20Z
smbd: Move downgrade_share_lease into downgrade_lease

The next step will simplify the logic of the code.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
cff1b31c by Volker Lendecke at 2018-10-02T20:22:37Z
smbd: Simplify downgrade_share_lease

Coalesce the NT_STATUS_OPLOCK_BREAK_IN_PROGRESS case into just one
if-condition

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Tue Oct  2 22:22:37 CEST 2018 on sn-devel-144

- - - - -
1853fd4f by Andreas Schneider at 2018-10-02T23:07:15Z
third_party: Update cmocka to version 1.1.3

* Added function to filter tests (cmocka_set_test_filter)
* Fixed fixture error reporting
* Some improvement for API documentation -> https://api.cmocka.org/
* Fixed subunit output on failures
* Do not abort if a test is skipped

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
965d7e90 by Volker Lendecke at 2018-10-02T23:07:15Z
includes: Remove an unused #define

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f8fab07c by Volker Lendecke at 2018-10-02T23:07:15Z
includes: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4060e263 by Volker Lendecke at 2018-10-02T23:07:15Z
includes: Remove an unused #define

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
826a2127 by Volker Lendecke at 2018-10-02T23:07:15Z
lib: Remove unused tdb_pack_append()

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e22d5e5b by Volker Lendecke at 2018-10-02T23:07:15Z
smbd: Fix a warning

gcc complains that the "const" is ignored on function return
types. Right now I'm compiling this file a lot, so silence this
warning :-)

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
43a0e635 by Volker Lendecke at 2018-10-03T02:11:59Z
leases: Streamline leases_db_key a bit

We don't need to talloc the blob, it's always the same size

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Oct  3 04:11:59 CEST 2018 on sn-devel-144

- - - - -
54207bb3 by Alexander Bokovoy at 2018-10-05T11:08:25Z
s4/auth/tests: Fix kerberos test string size

>>> len("user0 at samba.example.com")
23

But the string definition does not take a final '\0' into account.
As per Volker's suggestion, use compiler's support to allocate
the string properly.

Signed-off-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
6784ff24 by Martin Schwenke at 2018-10-05T14:16:29Z
ctdbd_conn: Generalise inaccurate error message

Seeing:

  ctdb_read_packet failed: Cannot allocate memory
  [...,  0] ../source3/lib/ctdbd_conn.c:121(cluster_fatal)
  cluster fatal event: ctdbd died

The error is due to a memory allocation failure rather than ctdbd
dying.  However, the error message makes people wonder why ctdbd died.

Another alternative would be to wrap cluster_fatal() and have the
wrapper interpret the return value from ctdb_read_packet() to choose
from a set of more precise messages to pass to cluster_fatal().  For a
memory allocation it isn't strictly necessary to call cluster_fatal(),
but all is probably lost and it is still probably better to try to
exit cleanly as soon as possible instead of crashing somewhere.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Fri Oct  5 16:16:29 CEST 2018 on sn-devel-144

- - - - -
71896fdd by Amitay Isaacs at 2018-10-08T00:46:20Z
ctdb-daemon: Add invalid_records flag to ctdb_db_context

If a node becomes INACTIVE, then all the records in volatile databases
are invalidated.  This avoids the need to include records from such
nodes during subsequent recovery after the node comes out INACTIVE state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
040401ca by Amitay Isaacs at 2018-10-08T00:46:20Z
ctdb-daemon: Don't pull any records if records are invalidated

This avoids unnecessary work during recovery to pull records from nodes
that were INACTIVE just before the recovery.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
c4ec99b1 by Amitay Isaacs at 2018-10-08T00:46:20Z
ctdb-daemon: Invalidate records if a node becomes INACTIVE

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
dcc99359 by Martin Schwenke at 2018-10-08T00:46:20Z
ctdb-tests: Add recovery record resurrection test for volatile databases

Ensure that deleted records and vacuumed records are not resurrected
from recently inactive nodes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
202b9027 by Amitay Isaacs at 2018-10-08T00:46:20Z
ctdb-vacuum: Simplify the deletion of vacuumed records

The 3-phase deletion of vacuumed records was introduced to overcome
the problem of record(s) resurrection during recovery.  This problem
is now handled by avoiding the records from recently INACTIVE nodes in
the recovery process.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
ef052397 by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-vacuum: Fix the incorrect counting of remote errors

If a node fails to delete a record in TRY_DELETE_RECORDS control during
vacuuming, then it's possible that other nodes also may fail to delete a
record.  So instead of deleting the record from RB tree on first failure,
keep track of the remote failures.

Update delete_list.remote_error and delete_list.left statistics only
once per record during the delete_record_traverse.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
e15cdc65 by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-vacuum: Remove unnecessary check for zero records in delete list

Since no records are deleted from RB tree during step 1, there is no
need for the check.  Run step 2 unconditionally.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
d18385ea by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-daemon: Drop implementation of RECEIVE_RECORDS control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
81dae71f by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-protocol: Mark RECEIVE_RECORDS control obsolete

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
2f89bd96 by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-protocol: Drop marshalling code for RECEIVE_RECORDS control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
83b3c567 by Amitay Isaacs at 2018-10-08T00:46:21Z
ctdb-tests: Drop code for RECEIVE_RECORDS control

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13641

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
a66a9693 by Martin Schwenke at 2018-10-08T00:46:21Z
ctdb-tests: Drop ps_ctdbd()

This was used for debugging tests by ensuring that the arguments to
ctdbd were as expected.  It no longer outputs anything useful because
ctdbd is now started without arguments.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f57e5bbd by Martin Schwenke at 2018-10-08T00:46:21Z
ctdb-tests: Rename ctdb_start_all() -> ctdb_init()

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4642a347 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Rename _ctdb_start_all() -> ctdb_start_all()

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f1ede41a by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Don't used daemons_start()/daemons_stop() directly in tests

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
90f6b0a1 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Drop functions daemons_start(), daemons_stop()

There are too many functions to start/stop daemons.  Simplify this.

Inline the functionality into ctdb_start_all() and ctdb_stop_all().

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2cd6a003 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Explicitly check for local daemons when shutting down

This is clearer if the logic is explicit...  and...

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f2e4a5e9 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Drop unused function maybe_stop_ctdb()

There are too many functions to start/stop daemons.  Simplify this.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
8bde6fa0 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Don't remove non-existent test database directory

This directory is no longer used.  Lack of removal doesn't seem to
cause a problem.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
a9ac3301 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Do not use ctdbd_wrapper in local daemon tests

Run the daemon directly and shut it down using ctdb shutdown.

The wrapper waits for ctdbd to reach >=FIRST_RECOVERY runstate within
a timeout period and shuts ctdbd down if that doesn't happen.  This is
only really used to ensure that ctdbd doesn't exit early after an
apparently successful start.  There are no known cases where ctdbd
will continue running but fail to reach >=FIRST_RECOVERY runstate.

When ctdbd is started in tests, the test code will wait until ctdbd is
in a healthy state on all nodes before proceeding, so there is
effectively no change in behaviour.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0eabac52 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Be more efficient about starting/stopping local daemons

Don't loop, just use onnode all.

For shutting down, use onnode -p all.  This results in a significant
time saving for stopping many deamons because "ctdb shutdown" is now
synchronous.

onnode -p all can be used to start daemons directly because they
daemonize.  However, this does not work under valgrind because the
valgrind process does not exit, so onnode will wait forever for it.
In this case, use onnode without the -p option.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
03dddc37 by Martin Schwenke at 2018-10-08T00:46:22Z
ctdb-tests: Don't format IPv4 octets as hex digits

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
36eb7388 by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Be more careful when building node addresses

The goal is to allow more local daemons by expanding the address range
rather than generating invalid addresses.

For IPv6, use all 4 trailing hex digits.

For IPv4, use the last 2 octets.  Although 127.0.0.0 is a /8 network,
avoid unexpected issues due to 0 and 255 in the last octet.  Use a
maximum of 100 addresses per "subnet" starting at .1.  Keep the first
group of addresses in 127.0.0.0/24 to continue to allow a reasonable
number of nodes to be tested with socket-wrapper.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0dfb3c87 by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Be more careful when building public IP addresses

The goal is to allow more local daemons by expanding the address range
rather than generating invalid addresses.

For IPv6, use a separate address space instead of an offset for the
2nd address.

For IPv4, use the last 2 octets with addresses starting at
192.168.100.1 and 192.168.200.1.  Avoid addresses with 0 and 255 in
the last octet by using a maximum of 100 addresses per "subnet"
starting at .1.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
6ac5124b by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Support closing of stdin in local daemons ssh stub

Not sure this is needed but this makes it behave the same as ssh.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2aa006a3 by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tools: Have onnode pass -n option even when regular ssh not in use

ONNODE_SSH is really a test hook, so it doesn't need to support
completely random values.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d67d8ed4 by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Shut down transaction_loop clients more cleanly

A transaction_loop client can exit with a transaction active when its
time limit expires.  This causes a recovery and causes problems with
the test cleanup, which detects unwanted recoveries and fails.

Set a flag when the time limit expires and exit cleanly before the
next transaction is started.

Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
52dcecbc by Martin Schwenke at 2018-10-08T00:46:23Z
ctdb-tests: Add extra debug to large database recovery test

This test sometimes fails, probably because the test is flakey.
Either the records aren't being added correctly or the counting of
records loses records.  Try to debug both possibilities.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
80f3f7c1 by Martin Schwenke at 2018-10-08T03:36:11Z
ctdb-tests: Improve counting of database records

Record counts are sometimes incomplete for large databases when
relevant tests are run on a real cluster.

This probably has something to do with ssh, pipes and buffering, so
move the filtering and counting to the remote end.  This means that
only the count comes across the pipe, instead of all the record data.

Instead of explicitly excluding the key for persistent database
sequence numbers, just exclude any key starting with '_'.  Such keys
are not used in tests.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Oct  8 05:36:11 CEST 2018 on sn-devel-144

- - - - -
f5c6bd5c by David Mulder at 2018-10-08T16:17:14Z
gpo: abstract methods are defined in the parent class

These methods don't need redefined in the child
class.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
5dddb784 by David Mulder at 2018-10-08T16:17:14Z
gpo: Initialize SamDB in the gp_sec_ext

The SamDB is only used by the gp_sec_ext, and
isn't needed elsewhere, so initialize it where
we need it and avoid passing it around
everywhere.
It makes the most sense to put this in the setter
class that uses it, so pass our creds down so we
have access to it, then initialize it there.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
aa9b07ba by David Mulder at 2018-10-08T16:17:14Z
gpo: gp_sec_ext should check whether to apply

Whether an extension should apply should be
determined by the extension, not by the
calling script.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
f702ad94 by David Mulder at 2018-10-08T16:17:15Z
gpupdate: Remove the unnecessary url parameter

The samdb object isn't initialized here anymore,
but in the gp_sec_ext, so this parameter to
gpupdate does nothing.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
4354071b by David Mulder at 2018-10-08T16:17:15Z
gpo: Initialize gp_ext variables in constructor

Initialize variables for the gp_ext in the
constructor instead of passing them via the parse
function.
This is a dependency of the "gpo: Implement
process_group_policy() gp_ext func" patch, since
the parse() function is now called by the ext,
instead of by gpupdate within apply_gp(). The
parse() function should only take the path
variable, to simplify writing Client Side
Extensions.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
fb22582a by David Mulder at 2018-10-08T16:17:15Z
gpo: Remove unused methods from gp_sec_ext

These functions were added by Luke, but have
never actually done anything. If/when we
read from these *.pol files, we won't need these
separate functions to do it.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
7bb326a6 by David Mulder at 2018-10-08T16:17:15Z
gpo: Implement process_group_policy() gp_ext func

MS spec describes the policy callback as a
function called ProcessGroupPolicy which accepts
a pDeletedGPOList and a pChangedGPOList param.
The Group Policy Client Side Extension then
iterates over the deleted, then the changed gpo
lists and applies/unapplies policy. We should do
this also.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
7cef6953 by David Mulder at 2018-10-08T16:17:15Z
gpo: Remove unused gp_ext.list() function

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
78601b35 by David Mulder at 2018-10-08T16:17:15Z
gpo: Move policy application to the gp_ext

Policy specific setting application should be
handled by the group policy extension, not the
read/parse handler.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
96ffc969 by David Mulder at 2018-10-08T16:17:15Z
gpo: remove unreached non-DC branch in gp_sec_ext.apply_map()

We don't get this far if we are not a DC, and if somehow we do the
errors will be no more informative due to this special case.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
8a21ae62 by David Mulder at 2018-10-08T16:17:15Z
gpo: apply_map should not be required for gp_ext

The apply_map function should not be a requirement
to implement the gp_ext class, since only the
gp_sec_ext uses it now.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
cc02de6b by David Mulder at 2018-10-08T16:17:15Z
gpo: Create a function for returning applied settings

This returns a list of guids for gpos applied
plus settings applied and their previous values.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
9d7a0bb3 by Douglas Bagnall at 2018-10-08T16:17:15Z
gpo: avoid quadratic behaviour in guid retrieval

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: David Mulder <dmulder at suse.com>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
30f6ac9c by David Mulder at 2018-10-08T16:17:15Z
gpo: Use the new process_group_policy() for unapply

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
cab16505 by David Mulder at 2018-10-08T16:17:15Z
gpo: Calculate deleted gpos and unapply them

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
7d07d8bf by David Mulder at 2018-10-08T16:17:16Z
gpo: add unapply to the gp_sec_ext

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
471089ee by David Mulder at 2018-10-08T16:17:16Z
gpo: Remove unused apply_log_pop() and list() funcs

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
4e98d18b by David Mulder at 2018-10-08T16:17:16Z
gpupdate: Add the --force option

This option forces the reapplication of policy,
and works the same as MS 'gpupdate /force'

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
6ac1445a by David Mulder at 2018-10-08T16:17:16Z
gpupdate: test the new --force option

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
ba87e23b by Douglas Bagnall at 2018-10-08T16:17:16Z
gpo PEP8: balance whitespace around equals

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: David Mulder <dmulder at suse.com>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
cb3eb79e by David Mulder at 2018-10-08T16:17:16Z
gpo: Test the new get_applied functions

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
f390dbfa by David Mulder at 2018-10-08T16:17:16Z
gpo: test the get_deleted_gpos_list() function

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

- - - - -
a5a4f979 by David Mulder at 2018-10-08T19:25:59Z
gpo: Test process_group_policy in gp_sec_ext

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Aurelien Aptel <aaptel at suse.com>

Autobuild-User(master): Aurélien Aptel <aaptel at samba.org>
Autobuild-Date(master): Mon Oct  8 21:25:59 CEST 2018 on sn-devel-144

- - - - -
8f211efb by Volker Lendecke at 2018-10-08T20:17:09Z
tdb_unpack: Convert to size_t for internal calculations

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0c0c79b3 by Volker Lendecke at 2018-10-08T20:17:09Z
tdb_unpack: Correct "len" arg for "B" format

All but one of the users of the "B" format specifier passed in a pointer
to uint32_t instead of what tdb_unpack expected, an "int". Because this
is a purely internal API, change the tdb_unpack function and adjust that
one caller.

To reviewers: Please check carefully, thanks :-)

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ba787dc4 by Volker Lendecke at 2018-10-08T20:17:09Z
tdb_unpack: Protect against overflow

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1fb2d10d by Volker Lendecke at 2018-10-08T20:17:09Z
registry: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2a35cbb0 by Volker Lendecke at 2018-10-08T20:17:09Z
registry: Add error checks to regdb_fetch_keys_internal

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a9ed0e93 by Volker Lendecke at 2018-10-08T20:17:09Z
registry: Add error checks to regdb_unpack_values

This makes "regdb_unpack_values" take a size_t as buflen. The only
caller calls it with TDB_DATA.dsize, which *is* size_t. Convert the
internal "len" variable to the unsigned size_t as well and add overflow
checks. This depends on tdb_unpack to either return -1 or a positive
value less than or equal to the passed-in "size_t" buflen;

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f20d018a by Volker Lendecke at 2018-10-08T20:17:10Z
registry: Print failure of regdb_unpack_values

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5ba7b204 by Volker Lendecke at 2018-10-08T20:17:10Z
registry: Don't use an uninitialized value

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b83763d1 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Add tdb_chainwalk_check

This captures the tdb_rescue protection against circular hash chains
with a slow pointer updated only on every other record traverse

If a hash chain has a loop, eventually the next_ptr
will cycle around and be identical to the 'slow' pointer.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e63f7bd3 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Make tdb_find circular-safe

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6502f7a3 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Make tdb_dump_chain circular-list safe

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e02c4a41 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Make tdb_find_dead circular-safe

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ade339c8 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Make get_hash_length circular-safe

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
75e79ca5 by Volker Lendecke at 2018-10-08T20:17:10Z
tdb: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7e1ad4c5 by Volker Lendecke at 2018-10-08T20:17:11Z
tdb: Make the freelist walk circular-safe

We can't really do the full check while the freelist is modified on the
fly. As long as we don't merge any freelist entries, we should be good
to apply this check.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
bdacc41f by Volker Lendecke at 2018-10-08T20:17:11Z
tdb: Basic test for circular hash chain fix

This just walks tdb_find by searching for a nonexistent record

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f36a538c by Volker Lendecke at 2018-10-08T20:17:11Z
tdb: Basic test for circular freelist fix

Try to store a record for which the (circular) freelist does not have
any entry.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e7d3678d by Volker Lendecke at 2018-10-08T20:17:11Z
lib: Avoid the use of open_memstream in tevent_req_profile_string

Solaris does not have it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13629
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2329518f by Volker Lendecke at 2018-10-08T20:17:11Z
pdb: Use "sid_compose" where appropriate

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f6137454 by Volker Lendecke at 2018-10-08T20:17:11Z
pdb: Fix some "(ret == true)" to just "(ret)"

"ret" is a boolean, so this should not change semantics

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d25f88f7 by Volker Lendecke at 2018-10-08T23:22:53Z
pdb: Reduce code duplication in make_user_info()

10 lines less and a few hundred (-O0) bytes .text less

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Oct  9 01:22:53 CEST 2018 on sn-devel-144

- - - - -
d46a5d63 by Volker Lendecke at 2018-10-09T20:32:12Z
build: Multi-line deps

We'll add one in the next commit

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9e26f6de by Volker Lendecke at 2018-10-09T20:32:12Z
auth: Use the zlib version of crc32

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dfa51233 by Volker Lendecke at 2018-10-09T20:32:12Z
drsuapi: Use the zlib version of crc32

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
37f6e2a3 by Volker Lendecke at 2018-10-09T23:41:52Z
lib: Remove lib/crypto/crc32.[ch]

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Oct 10 01:41:52 CEST 2018 on sn-devel-144

- - - - -
ea6421d7 by Joe Guo at 2018-10-10T04:16:21Z
emulate/traffic: allow traffic_replay to run users and groups generate multiple times

When we run `traffic_replay --generate-users-only`, if we cancel it or
it breaks in middle, it won't do anything when we try to run it again.

This is because the code will check the first user/group to create. If
it's already there, then it thought task already done, and break the loop.

This commit change the behavior:
We search existing users/groups first, skip existing ones, and
create non-existing ones. So we can run it multi-times to make sure the
expected users and groups are actually created.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e5cf5a68 by Joe Guo at 2018-10-10T04:16:21Z
samba/logger: add logger module for python

We need a consitent way for logging in Samba Python code.

This module provides a factory method `get_samba_logger` to create logger,
with a reasonable default format and optional color.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e8a1773c by Joe Guo at 2018-10-10T04:16:21Z
emulate/traffic: apply new logger to replace print

These print are actually progress infomation, should use logger to
print to stderr, other than stdout.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
33ce1fa2 by Joe Guo at 2018-10-10T04:16:21Z
script/traffic_replay: print packets data to stderr

This is debug info, should print to stderr.
Otherwise it will flood stdout.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
57594c87 by Joe Guo at 2018-10-10T04:16:21Z
script/traffic_replay: get debug level via api

The -d option will set samba global debug level automatically.
We should not parse and use the passed in value.

Use samba.get_debug_level instead.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4260fb87 by Joe Guo at 2018-10-10T04:16:21Z
script/traffic_replay: apply new logger to replace print

Use logger to replace print

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0979d960 by Joe Guo at 2018-10-10T04:16:22Z
netcmd: modify Command.get_logger to use get_samba_logger

By doing this, we don't need to repeat the log level convert code any more.
Also, logs have colors now.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
126a14b4 by Joe Guo at 2018-10-10T04:16:22Z
netcmd: apply the new get_logger to cmds

This is an example of how to use the new logger.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
473c1c00 by Douglas Bagnall at 2018-10-10T04:16:22Z
popt_common_creds: actually use the ignore_missing_conf flag

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Pair-programmed-with: Olly Betts <olly at survex.com>
Reviewed-by: Noel Power <nopower at suse.com>

- - - - -
77ef686f by Gary Lockyer at 2018-10-10T04:16:22Z
pyauth: Remove imessaging_ctx parameter to new

The pyauth code assumes the messaging context code is a py_talloc
object.  But the code in pymessaging returns a wrapped talloc object.
Removing the parameter as it's not currently used by any code.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Noel Power <nopower at suse.com>

- - - - -
79e1ed1f by Douglas Bagnall at 2018-10-10T04:16:22Z
python/upgradehelpers: use int not long for PY3

int works OK for py2 also.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <nopower at suse.com>

- - - - -
131f98ed by Douglas Bagnall at 2018-10-10T07:28:20Z
tests/python/ldap: use int instead of long for time_t

Python int is at least a C long; Python long disappears in Py3.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <nopower at suse.com>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Wed Oct 10 09:28:20 CEST 2018 on sn-devel-144

- - - - -
8b972848 by Ralph Boehme at 2018-10-10T20:22:12Z
s4:torture: FinderInfo conversion test with AppleDouble without xattr data

This testcase demonstrates that the AppleDouble conversion in vfs_fruit
doesn't correctly convert the FinderInfo data from the AppleDouble file
to a stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7e010abb by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: fix two comments

Thanks to the recent addition of ad_convert_xattr() we now correctly
handle this case.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8ee7e613 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: store filler bytes from AppleDouble file header in struct adouble

This can later be used to distinguish between macOS created AppleDouble
files and AppleDouble files created by Samba or Netatalk.

macOS:    "Mac OS X        "
Samba:    "Netatalk        "
Netatalk: "Netatalk        "

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d161e047 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: move setting ADEID_FINDERI length to ad_convert_xattr()

ad_convert_xattr() does the conversion of the xattr data in the
AppleDouble file, so we should update it's size there and should not
defer it to the caller.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8bc36d72 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: do direct return from error checks in ad_convert()

Subsequent commits will move the mmap() into the subfunctions. This
change just prepares for that.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
99cc9ef8 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: remove unneeded fd argument from ad_convert()

Use the struct adouble member ad_fd instead of passing it as an
argument. Who did that in the first place? :)

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b355a095 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: move storing of modified struct adouble to ad_convert()

ad_convert() modified it, so let ad_convert() also save it to disk. No
change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d27d0326 by Ralph Boehme at 2018-10-10T20:22:13Z
vfs_fruit: move FinderInfo conversion to helper function and call it from ad_convert()

No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4f1174b6 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: move FinderInfo lenght check to ad_convert()

The final step in consolidating all conversion related work in
ad_convert(). No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
98bd7c0a by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: split out truncating from ad_convert()

This may look a little ill-advised as this increases line count, but
the goal here is modularizing ad_convert() itself and making it as slick
as possible helps achieving that goal.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b948681b by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: use ADEDOFF_RFORK_DOT_UND offset macro in ad_convert_truncate()

We really want the fixed size offset here, not a calculated one. Note
that "ad_getentryoff(ad, ADEID_RFORK)" is equal to ADEDOFF_RFORK_DOT_UND
in this case.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4c7e1de4 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: split out moving of the resource fork

No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
93b7e056 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: use ADEDOFF_RFORK_DOT_UND offset macro in ad_convert_move_reso()

We really want the fixed size offset here, not a calculated one. Note
that "ad_getentryoff(ad, ADEID_FINDERI) + ADEDLEN_FINDERI" is equal to
ADEDOFF_RFORK_DOT_UND.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f91e0c85 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: fix error returns in ad_convert_xattr()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
918c6c59 by Ralph Boehme at 2018-10-10T20:22:14Z
vfs_fruit: let the ad_convert_*() subfunctions mmap as needed

This may mean that we mmap twice when we convert an AppleDouble file,
but this is the only sane way to cleanly modularize ad_convert().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1692ca5f by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: let the ad_convert_*() subfunction update the on-disk AppleDoube header as needed

Another step in simplifying ad_convert() itself. It means that we may
write to disk twice, but is only ever done once per AppleDouble file.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
70d3ae5a by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: call ad_convert_move_reso() from ad_convert_xattr()

ad_convert_xattr() is the place that triggers the need to move the
resource fork, so it should also call ad_convert_move_reso().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5598e6bc by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: add check for OS X filler in FinderInfo conversion

This ensures that the function only acts on AppleDouble files created by
macOS and not AppleDouble files created by us that are already in the
correct format (only using the Resource Fork).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
acb72c1e by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: add out arg "converted_xattr" to ad_convert_xattr

Used to let the caller know if a conversion has been done. Currently not
used in the caller, that comes next.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9cf087a4 by Ralph Boehme at 2018-10-10T20:22:15Z
vfs_fruit: make call to ad_convert_truncate() optional

Call ad_convert_truncate() based on whether the previous call
ad_convert_xattr() returned converted_xattr=true.

Upcoming fixes for a different Samba bug (#13642) will hook into calling
ad_convert_truncate() in other cases, this also prepares for that.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
31daab88 by Ralph Boehme at 2018-10-10T23:30:13Z
vfs_fruit: move check in ad_convert() to ad_convert_*() subfunctions

Currently the whole conversion is skipped if the FinderInfo entry in the
AppleDouble file is of the default size (ie not containing xattrs).

That also means we never converted FinderInfo from the AppleDouble file
to stream format. This change finally fixes this.

Note that this keeps failing with streams_depot, much like the existing
known-fail of "samba3.vfs.fruit streams_depot.OS X AppleDouble file
conversion". Fixing the conversion to work with vfs_streams_depot is a
task for another day.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Thu Oct 11 01:30:13 CEST 2018 on sn-devel-144

- - - - -
96b5bf13 by Björn Baumbach at 2018-10-11T08:28:17Z
auth: move copy_session_info() from source3 into the global auth context

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
ea38be48 by Björn Baumbach at 2018-10-11T08:28:17Z
python: Add samba.auth.copy_session_info()

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
9a44be63 by Björn Baumbach at 2018-10-11T08:28:17Z
s4-auth: fix a typo in a comment

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
29af2df4 by Björn Baumbach at 2018-10-11T08:28:17Z
s4-auth: use TALLOC_FREE() shortcut

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
29e757ac by Björn Baumbach at 2018-10-11T08:28:18Z
s4-auth: fetch possible out of memory error

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
f3b7ba17 by Björn Baumbach at 2018-10-11T08:28:18Z
s4-auth: allow to create unix token from system session info

Without this patch security_token_to_unix_token() fails with
NT_STATUS_ACCESS_DENIED, because the system session does only
have one SID.
For a typical token are at least two or more SIDs expected.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
6f08cb66 by Björn Baumbach at 2018-10-11T08:28:18Z
s4-auth: allow to pass original_user_name=NULL to auth_session_info_fill_unix()

With this patch the auth_session_info_fill_unix() uses the "unix_name"
from the session_info->unix_info if no original_user_name was specified.

This is used to process a system session info where no original_user_name
is given.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
ab558fa1 by Björn Baumbach at 2018-10-11T08:28:18Z
pysmbd: add option to pass a session info to set_nt_acl() function

A filled session info is needed by some vfs modules, e.g. full_audit.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
bc8d0d51 by Björn Baumbach at 2018-10-11T08:28:18Z
pysmbd: handle file not found error

Avoid PANIC: internal error

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
38fe315b by Björn Baumbach at 2018-10-11T08:28:18Z
samba-tool ntacl: pass system session to get/set-ntacl functions

The filled session is needed in different vfs modules.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
01ff09ad by Björn Baumbach at 2018-10-11T08:28:18Z
s3/py_passdb: add get_domain_sid() to get domain sid from secrets database

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
e54d4ffb by Björn Baumbach at 2018-10-11T08:28:18Z
samba-tool ntacl: allow to run get/set-ntacl command in non-AD-DC role

Can be used to get and apply NT-ACLs on Samba member servers.

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
e5786276 by Björn Baumbach at 2018-10-11T08:28:19Z
selftest: test samba-tool ntacl get/set on AD member server

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
b773be33 by Björn Baumbach at 2018-10-11T08:28:19Z
dns update: add missing newline in error debug message

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
ff3e2fa8 by Björn Baumbach at 2018-10-11T11:40:27Z
vfs_full_audit: ntimes: log a-, m-, c- and creation-time

Signed-off-by: Björn Baumbach <bb at sernet.de>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Björn Baumbach <bb at sernet.de>
Autobuild-Date(master): Thu Oct 11 13:40:27 CEST 2018 on sn-devel-144

- - - - -
cb5ad7fe by Douglas Bagnall at 2018-10-12T02:16:21Z
s4/script/samba_upgradeprovision: use int not long for Python 3

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1851c35e by Douglas Bagnall at 2018-10-12T02:16:21Z
s4/script/samba_upgradeprovision: remove duplicate (contradictory) dict key

The second, winning, entry says '"defaultSecurityDescriptor": replace + add'

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
69fad8bf by Douglas Bagnall at 2018-10-12T02:16:22Z
s4/script/samba_upgradeprovision: remove unused variable

A similarly named variable is always set two lines down, so we don't need this

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1139a4a6 by Douglas Bagnall at 2018-10-12T02:16:22Z
s4/script/samba_upgradeprovision: set global dnNotToRecalculateFound var

as probably intended. Without this the local variable shadows the
global one and is never used while the global one is never changed.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
dec3eda1 by Tim Beale at 2018-10-12T02:16:22Z
tests: Add corner-case test: fromServer points to dead server

The fromServer attribute is slightly unique, in that it's a DN (similar
to a one-way link), but it is also a mandatory attribute.

Currently, if fromServer gets a bad value (i.e. a dead server that has
been expunged), the DSDB rejects any attempts to modify the associated
nTDSConnection object (regardless of whether or not you're actually
changing the fromServer attribute).

This patch adds a test-case that demonstrates how the DB can get into
such a state.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4092b369 by Andrew Bartlett at 2018-10-12T02:16:22Z
dsdb: Ensure that a DN (now) pointing at a deleted object counts for objectclass-based MUST

Add the 'reveal_internals' controls when performing objectclass-based
checks of mandatory attributes. This prevents the extended_dn DSDB
module from suppressing attributes that point to deleted (i.e.
non-existent/expunged) objects.

This ensures that, when modifying an object (and often not even
touching the mandatory attribute) that the fact that an attribute is a
DN, and the DN target is deleted, that the schema check will still pass.

Otherwise a fromServer pointing at a dead server can cause failures,
i.e. you can't modify the affected object at all, because the DSDB
thinks a mandatory attribute is missing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13621

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
24669e57 by Tim Beale at 2018-10-12T02:16:22Z
dsdb: Remove redundant variable/check

Previously, this code used to live inside the loop, so the
checked_reveal_control was needed to save ourselves unnecessary work.

However, now that the code has been moved outside the loop, the
checked_reveal_control variable is just unnecessary complication.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
b2c4b829 by Tim Beale at 2018-10-12T02:16:22Z
netcmd: Change Py3 incompatible long() for tombstone expunge

The code to expunge tombstones uses long(), which is not Python3
compatible. Python3 uses int() instead, and works out how big it needs
to be.

As long as we don't run the samba-tool command on a 32-bit machine
after the year 2038, then we should avoid any integer overflow on
Python 2.x.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3efb4588 by Tim Beale at 2018-10-12T05:23:26Z
dsdb: Add dsdb_request_has_control() helper function

Most of the DSDB modules only want to check the existence of a control,
rather than access the control itself. Adding a helper function allows
the code to ask more natural-sounding yes/no questions, and tidies up
an ugly-looking long-line in extended_dn_out.c.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Oct 12 07:23:26 CEST 2018 on sn-devel-144

- - - - -
2fc855e7 by Douglas Bagnall at 2018-10-12T13:27:07Z
samba-tool drs showrepl: do not crash if no dnsHostName found

This should not happen, but it does sometimes in an autobuild
environment. Rather than reporting this by crashing, we report it by
showing there is no DNS name.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Oct 12 15:27:07 CEST 2018 on sn-devel-144

- - - - -
84615c19 by Andreas Schneider at 2018-10-16T06:42:18Z
replace: Add memset_s() if not available

See https://en.cppreference.com/w/c/string/byte/memset

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9291a333 by Andreas Schneider at 2018-10-16T09:38:40Z
s3:lib:popt: Use memset_s() to burn password string

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Tue Oct 16 11:38:40 CEST 2018 on sn-devel-144

- - - - -
fb573c68 by Volker Lendecke at 2018-10-16T16:00:10Z
winbindd_cache: Fix timeout calculation for sid<->name cache

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fdb50817 by Volker Lendecke at 2018-10-16T16:00:10Z
namemap_cache: Absorb the expired calculation into namemap_cache.c

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2557ae53 by Volker Lendecke at 2018-10-16T19:20:19Z
lib: Move the "expired" for gencache_parse calculation into gencache.c

Make it more robust

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Oct 16 21:20:19 CEST 2018 on sn-devel-144

- - - - -
3713905f by Aaron Haslett at 2018-10-17T03:40:06Z
dns: dlz_bind9 reference count logging

dlz_bind9 has to count the number of times the plugin is 'created' by bind's
plugin manager so it doesn't repeat setup.  Logging doesn't reflect this
reference counting logic properly and so messages like "samba_dlz: shutdown"
can, confusingly, come up when the database connection has not actually been
severed.  This patch adds the necessary logging.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13655
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
539daefa by Tim Beale at 2018-10-17T03:40:07Z
libnet/drs: Update replication debug to report link progress

Update the replication debug (for joins/backups) so that it's easier to
see how far through syncing the links we are. E.g. with 150,000 links,
you just get screeds of debug like this, with no real idea how far
through the replication is.

Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[1500/150024]
Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[1500/150024]
Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[1500/150024]

This patch now applies to links the same debug logic we use for objects,
and changes it to look like:

Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[57024/150024]
Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[58524/150024]
Partition[DC=addom,DC=samba,DC=example,DC=com] objects[11816/11720]
linked_values[60024/150024]

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fdb6b86b by Tim Beale at 2018-10-17T06:56:42Z
drs_util: Improve memory usage when joining large DB

drs_Replicate.replicate() could consume a large amount of memory when
replicating a large DB. This is not a leak - the memory gets freed when
the function returns (i.e. once the partition is fully replicated).
However, while the partition is in the process of being replicated, it
accumulates memory for each replication chunk it receives. This can have
considerable overhead with 1000s of objects/links in the partition.

This was exhausting memory when joining a VM with 1Gb RAM to a DC with
25K users (average ~15 group memberships per user).

It seems that by storing a reference to something that's on the ctr's
talloc tree, it doesn't free up the memory for each ctr message (until
the function actually returns and req is destroyed).

With 10K users (and average 15 group memberships per user), .replicate()
consumed 211Mb of memory, according to talloc.report_full(). With this
patch, it goes down to just the current ctr message (1-2Mb).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Oct 17 08:56:42 CEST 2018 on sn-devel-144

- - - - -
410ec70b by Volker Lendecke at 2018-10-17T17:22:18Z
netsamlogon_cache: Fix talloc_stackframe error return leaks

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9b6166f7 by Volker Lendecke at 2018-10-17T17:22:19Z
netsamlogon_cache: Use "goto fail", save some lines

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
030a3e50 by Volker Lendecke at 2018-10-17T17:22:19Z
netsamlogon_cache: Add some error checks

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fa1f9593 by Volker Lendecke at 2018-10-17T17:22:19Z
netsamlogon_cache: Improve a DBG message

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4df055bb by Volker Lendecke at 2018-10-17T17:22:19Z
auth3: Avoid an explicit ZERO_STRUCT

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2a29ffc3 by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Avoid counting characters manually

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
140a0e05 by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Swap tests: Do cheapest first

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6007c444 by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Call string_term_tdb_data() only once

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
34fe8b1a by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Make gencache_pull_timeout a bit more robust

The previous version assumed a well-formed "val", we just handed it to
strtol without properly checking that it contains the delimiter. So
strtol could well run off the end of "val" in case of data corruption.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
78b8b916 by Volker Lendecke at 2018-10-17T17:22:19Z
gencache: Make gencache_pull_timeout return a payload DATA_BLOB

Both relevant callers created one anyway.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0817d107 by Volker Lendecke at 2018-10-17T17:22:20Z
gencache: Remove a redundant check

gencache_pull_timeout checks for NULL ptr already

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
85ec8644 by Volker Lendecke at 2018-10-17T20:22:51Z
gencache: Remove a redundant check

tdb_storev itself is robust against overflow due to multiple buffers

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Oct 17 22:22:51 CEST 2018 on sn-devel-144

- - - - -
d8ea16a3 by Tim Beale at 2018-10-18T04:15:24Z
join: LDAP connection to remote DC can timeout in large join

When joining a very large domain (e.g. 100K users), the replication can
take so long that the LDAP connection to the remote DC times out.

This patch avoids the problem by adding in a sanity-check after the
replication finishes that the LDB connection is still alive. If not,
then we reconnect.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13612

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
30277feb by Andrew Bartlett at 2018-10-18T04:15:24Z
join: Avoid searching for more than strictly required during sanity check

We check for the default base DN as this does require authentication, but
we do not need to search for more than just that (so use SCOPE_BASE) and
we need no attributes, so ask for none

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
10a9cc44 by Tim Beale at 2018-10-18T04:15:24Z
join: Sanity-check LDB connection before failed join cleanup

Joining a large DB can take so long that the LDAP connection times out.
The previous patch fixed the 'happy case' where the join succeeds.
However, if the commit or replication fails (throwing an exception),
then the cleanup code can also fail when it tries to delete objects from
the remote DC. This then gives you an error pointing to
cleanup_old_accounts() rather than what actually went wrong.

This patch adds a sanity-check that if the join fails, that the LDB
connection to the remote DC is still alive, before we start deleting
objects.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13612

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
da875eda by Joe Guo at 2018-10-18T04:15:24Z
uptodateness: add new module and migrate functions from visualize

Both visualize and drs cmd will have uptodateness functions.
Create a new module to reuse code.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
6ed82c7c by Joe Guo at 2018-10-18T04:15:24Z
uptodateness: migrate more methods from visualize

Move methods from cmd_uptodateness to new module.
Will reuse in drs cmd later.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
ff311f1d by Joe Guo at 2018-10-18T04:15:24Z
netcmd/visualize: rm unused code line

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
ac1fba31 by Joe Guo at 2018-10-18T04:15:24Z
uptodateness: extract function get_utdv_edges

Extract function to reuse later.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
bbef7d67 by Joe Guo at 2018-10-18T04:15:25Z
uptodateness: extract function get_utdv_distances

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
90975eb0 by Joe Guo at 2018-10-18T04:15:25Z
uptodateness: extract get_utdv_max_distance

To avoid returning 2 values from get_utdv_distances.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
74f7080b by Joe Guo at 2018-10-18T04:15:25Z
uptodateness: migrate get_kcc_and_dsas as a function

We need to reuse it in drs cmd.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
f4f5f174 by Joe Guo at 2018-10-18T04:15:25Z
uptodateness: add get_utdv_summary function

Get utdv summary from distances matrix and support attr filters.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
219b52cf by Joe Guo at 2018-10-18T04:15:25Z
netcmd/drs: add cmd_drs_uptodateness with json support

Add cmd to print uptodateness summary with json support.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

- - - - -
e404b6aa by Joe Guo at 2018-10-18T08:02:19Z
selftest: add tests for samba-tool drs uptodateness

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13658

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Oct 18 10:02:19 CEST 2018 on sn-devel-144

- - - - -
204bd0e4 by Douglas Bagnall at 2018-10-18T08:04:02Z
py3/tests/kcc: turn error into failure for flapping.d/kcc

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
30e9c00f by Douglas Bagnall at 2018-10-18T08:04:02Z
py3_tests/kcc : test_verify can hit KCCError as well as GraphError

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ce518f38 by Douglas Bagnall at 2018-10-18T11:17:30Z
ldb_ldif: be less horribly efficient in debugging

perf said all the time was in strlen.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Thu Oct 18 13:17:30 CEST 2018 on sn-devel-144

- - - - -
a0bad136 by Douglas Bagnall at 2018-10-19T01:43:58Z
ldb_ldif: avoid strlen(NULL)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Fri Oct 19 03:43:58 CEST 2018 on sn-devel-144

- - - - -
b6e45fb4 by Gary Lockyer at 2018-10-19T04:17:25Z
python tests Blackbox: add random_password

Add the random_password method to the BlackboxTestCase class and remove
duplicated copies from other test cases. Also use SystemRandom so that
the generated passwords are more cryptographically sound.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e1eee614 by Gary Lockyer at 2018-10-19T04:17:25Z
dsdb encrypted_secrets tests: Allow "ldb://" in file path

When creating a new user and specifying the local file path of the
sam.ldb DB, it's possible to create an account that you can't actually
login with.

This commit contains tests to verify the bug.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13653

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7b59cd74 by Gary Lockyer at 2018-10-19T07:34:46Z
dsdb encrypted_secrets: Allow "ldb:// and "mdb://" in file path

Correctly handle "ldb://" and "mdb://" schemes in the file path when
determining the path for the encrypted secrets key file.

When creating a new user and specifying the local file path of the
sam.ldb DB, it was possible to create an account that you could not
login with. The path for the key file was incorrectly calculated
for the "ldb://" and "mdb://" schemes, the scheme was not stripped from
the path and the subsequent open of the key file failed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13653

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Oct 19 09:34:46 CEST 2018 on sn-devel-144

- - - - -
e310ad7e by Philipp Gesang at 2018-10-19T11:59:04Z
s3:secrets: clean up sid before storing

SIDs may contain non-zero memory beyond SubAuthorityCount:

    {
    key(15) = "SECRETS/SID/FOO"
    data(68) = "\01\04\00\00\00\00\00\05\15\00\00\00}u@\8C\08\A3\06nx\95\16\FE\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00`F\92\B7\03\00\00\00\18e\92\B7\03\00\00\00 at H\92\B7\00\00\00\00"
    }

These parts are lost when converting to ``string format syntax``
so a roundtrip conversion does not result in the same binary
representation.

Ensure that these never reach the tdb by using an initialized
copy. This allows bitwise comparisons of secrets.tdb after
dumping SIDs as text and reading them back.

Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Oct 19 13:59:04 CEST 2018 on sn-devel-144

- - - - -
5b2c3f2f by Volker Lendecke at 2018-10-19T16:52:50Z
lib: Remove gencache.h from proto.h

It's a pain to recompile the world if gencache.h changes

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Oct 19 18:52:50 CEST 2018 on sn-devel-144

- - - - -
9e03fb60 by David Disseldorp at 2018-10-19T21:11:26Z
talloc: deprecate talloc_set_memlimit()

The memlimit functionality was never utilized by Samba. It adds unneeded
complexity, so flag it as deprecated.

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2679dd0f by Andreas Schneider at 2018-10-19T21:11:26Z
s3:registry: Avoid a double-free in reg_perfcount

Found by covscan.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f040d660 by Andreas Schneider at 2018-10-19T21:11:26Z
ndr: Init variables of GUID_from_data_blob()

Found by covscan.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1ca21b98 by Andreas Schneider at 2018-10-19T21:11:26Z
s4:torture: Fix the scope of the req variable in drsuapi test

Found by covscan.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d15a00ba by Andreas Schneider at 2018-10-20T00:17:56Z
s3:smbcontrol: Simplify the return code check

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Oct 20 02:17:56 CEST 2018 on sn-devel-144

- - - - -
a3d12252 by Martin Schwenke at 2018-10-22T04:04:20Z
ctdb-daemon: Return early when refusing to run an event script

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c9e1603a by Martin Schwenke at 2018-10-22T04:04:20Z
ctdb-daemon: Exit if eventd goes away

ctdbd enters a broken state if eventd goes away.  A clean shutdown is
not possible because that involves running events.  Restarting eventd
is possible but this might mask a serious problem and it is possible
that eventd might keep on disappearing.  Just exit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
80549927 by Amitay Isaacs at 2018-10-22T04:04:20Z
ctdb-common: Set close-on-exec for startup fd

The startup_fd should not be propagated to the child processes created
from a daemon.  It should only be used in the daemon code to return the
status of the startup.  Another use of startup_fd is to notify the
parent if the daemon process has exited.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
a1909603 by Amitay Isaacs at 2018-10-22T04:04:20Z
ctdb-event: Check the return status of sock_daemon_set_startup_fd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
fbea9d36 by Martin Schwenke at 2018-10-22T07:27:15Z
ctdb-daemon: Fix valgrind hit in event code

==25741== Syscall param write(buf) points to uninitialised byte(s)
==25741==    at 0x4939291: write (write.c:27)
==25741==    by 0x4868285: sys_write (sys_rw.c:68)
==25741==    by 0x13915D: sock_queue_trigger (sock_io.c:316)
==25741==    by 0x4DE6478: tevent_common_invoke_immediate_handler (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DE64A2: tevent_common_loop_immediate (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DEBE5A: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DEA2D6: ??? (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x4DE57E3: _tevent_loop_once (in /usr/lib/x86_64-linux-gnu/libtevent.so.0.9.37)
==25741==    by 0x15D1BA: ctdb_event_script_args (eventscript.c:821)
==25741==    by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315)
==25741==    by 0x110642: main (ctdbd.c:393)
==25741==  Address 0x57888a4 is 100 bytes inside a block of size 144 alloc'd
==25741==    at 0x48357BF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25741==    by 0x4B9B7C0: talloc_named_const (in /usr/lib/x86_64-linux-gnu/libtalloc.so.2.1.14)
==25741==    by 0x15CCC6: eventd_client_write (eventscript.c:430)
==25741==    by 0x15CCC6: eventd_client_run (eventscript.c:556)
==25741==    by 0x15CCC6: ctdb_event_script_run (eventscript.c:649)
==25741==    by 0x15D198: ctdb_event_script_args (eventscript.c:812)
==25741==    by 0x13B437: ctdb_start_daemon (ctdb_daemon.c:1315)
==25741==    by 0x110642: main (ctdbd.c:393)
==25741==

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13659

Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Oct 22 09:27:15 CEST 2018 on sn-devel-144

- - - - -
d114ca1e by Andrew Bartlett at 2018-10-23T03:50:24Z
selftest: Add expected-value testing for userParameters

This does not means that bugs like https://bugzilla.samba.org/show_bug.cgi?id=11881
are fixed, however we do not wish to cause further issues
without noticing it, eg during python3 fixes for dbcheck.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
f5f8676b by Noel Power at 2018-10-23T03:50:24Z
s4/scripting/bin: PY3 Make sure print statements are enclosed by '()'

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ff3f5411 by Noel Power at 2018-10-23T03:50:24Z
python/samba/tests: PY3 iterable has no sort method

map in python3 returns an iterable, in python2 it returned
a list. Iterable has no sort method, use sort function instead or
construct a list from the iterable so you can use list.sort

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
842f09aa by Noel Power at 2018-10-23T03:50:24Z
samba-tool: PY3 dict view doesn't have sort method,

Can't sort a dict view, create a list from view then use list.sort
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
41200262 by Noel Power at 2018-10-23T03:50:24Z
python/samba/gp_parse: PY3 fdeploy_sids needs to use key method for sort

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dcf787b2 by Noel Power at 2018-10-23T03:50:24Z
s4/scripting: PY3 need to convert cmp funct to key func for sort.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e5382179 by Noel Power at 2018-10-23T03:50:24Z
PY3: In a pure PY3 build filter-subunit was getting called without 'python'

tests were getting called with "| ${src}/selftest/filter-subunit" which
resulted in filter-subunit getting execve'd without a calling python. This
resulted in /usr/bin/python (default python) getting called and subsequent
imports failing.

- - - - -
9f8a570f by Noel Power at 2018-10-23T03:50:25Z
selftest/target: Make sure samba-tool is called with ${PYTHON}

Ensure python scripts are called with the python version that
is defined by $PYTHON

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c8403f33 by Noel Power at 2018-10-23T03:50:25Z
python/samba/tests: make sure samba-tool is called with ${PYTHON}

Ensure python scripts are called with the python version that
is defined by $PYTHON

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c751013f by Noel Power at 2018-10-23T03:50:25Z
testprogs/blackbox: Use PYTHON env variable for calling python scripts

Ensure samba-tool is called with correct python that is
defined by $PYTHON

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
15b2ab04 by Noel Power at 2018-10-23T03:50:25Z
s4/scripting: PY3 Ensure python scripts are run with correct python ver.

As part of port samba4.blackbox.samba3dump to python2/3
make sure test_samba3dump.sh runs samba3dump with $PYTHON which should
define the correct python version.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ee595e9b by Noel Power at 2018-10-23T03:50:25Z
s4/setup/tests: PY3 samba-tool needs to be called with correct python ver.

Ensure samba-tool python version defined by $PYTHON

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ade47b3d by Noel Power at 2018-10-23T03:50:25Z
PY3: Only decode when necessary

- - - - -
9ea6cb18 by Noel Power at 2018-10-23T03:50:25Z
python/samba/netcmd: PY3 only possible to decode bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
95dbe117 by Noel Power at 2018-10-23T03:50:25Z
python/samba/tests: PY2/PY3 port samba.tests.dcerpc.integer

Python3 no longer has a long type so the 'L' postfix is no
longer valid. Additionally in python2 an int that exceeds will
be transparently converted into a long when necessary

- - - - -
23239727 by Noel Power at 2018-10-23T03:50:26Z
python: py_strcasecmp_m & py_strstr_m don't handle unicode properly

py_strcasecmp_m & py_strstr_m use PyArg_ParseTuple() with 's' which
in Py2 tries to decode string with the default (e.g. ascii) encoding

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8d7830b6 by Noel Power at 2018-10-23T03:50:26Z
PY3: convert samba.tests.strings to Py2/Py3

Previously the py2 api for strcasecmp_m/strstr_m required strings/unicode
 but couldn't actually handle unicode with anything other than the default
encoding (e.g. ascii). The c-api as been fixed and the encoding steps
(which were unnecessary and causing errors in PY3) have been removed

- - - - -
600fde58 by Noel Power at 2018-10-23T03:50:26Z
python/samba: add alias for ConfigParser for PY2/PY3 compatability

ConfigParser module changed name to configParser in PY3, additionally
the behaviour regarding interpolation has changed. ConfigParser now
has a default interpolation param whose behaviour demands that '%' is
escaped. To maintain behaviour with the python2 version this default
param needs to be changed. Add some alias(s) and 'shim' Configparser
symbol in samba.compat to cater for this.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
04c118cf by Noel Power at 2018-10-23T03:50:26Z
python/samba/gp_parse: PY2/PY3 compat porting for gp_init.py

Fixes
1) use compat versions of ConfigParser and StringIO
2) open file needs to be opened in binary mode as write_pretty_xml
   routine uses BytesIO() object.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
72c5b270 by Noel Power at 2018-10-23T03:50:26Z
python/samba: use PY3 version of ConfigParser

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4287d119 by Noel Power at 2018-10-23T03:50:26Z
PY3: port samba.tests.samba3sam

- - - - -
9ae5fd65 by Noel Power at 2018-10-23T03:50:26Z
s4/scripting/bin: PY3 Fix exception tuple assignments.

In Python3 to access the exception arguments you need to now use
Exception.args, in Python2 you could access these direcly with the
'except' declaration.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ca12342b by Noel Power at 2018-10-23T03:50:26Z
PY3: relative import fixes

- - - - -
8eae420f by Noel Power at 2018-10-23T03:50:27Z
PY3: ensure StringIO usage is py2/py3 compatible

- - - - -
17728c2f by Noel Power at 2018-10-23T03:50:27Z
python/samba: ldb attribute string fix for wafsamba.tests

- - - - -
e5cac2b8 by Noel Power at 2018-10-23T03:50:27Z
python/samba: misc use of str for ldb.bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
370dcbb7 by Noel Power at 2018-10-23T03:50:27Z
samba_tool: Enclose iterator with list

Really strange bug caused by map being updated while being iterated.
This caused keys to be skipped and inconsistent and incorrect
results from ldapcmp.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
557ff7b7 by Noel Power at 2018-10-23T03:50:27Z
python/samba: samba4.blackbox.dbcheck.release-4-1-0rc3

* Various string related fixed py2/py3
* Fix strange double decode followed by encode code (see comments
  in change)
* Added dump_attr_values, simply printing attribute values (from
  sequence) doesn't work (when using string '%s' format codes in
  existing string). We need to print out string from bytes in PY3
  and fallback to repr(which will print b'blah') if we get a decode error

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0febd003 by Noel Power at 2018-10-23T03:50:27Z
s4/scripting: PY2/PY3 port for samba4.blackbox.upgradeprovision.current

o Fix various ldb attribute that need to be converted to string
o dict has no 'has_key' method
o ndr_unpack needs bytes not string
o b64encode needs bytes (so open file with binary mode)
o StandardError was removed in python3 use Exception instead
o fix octal literals
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f94a8137 by Noel Power at 2018-10-23T03:50:27Z
s4/scripting: Py2/Py3 for samba4.blackbox.upgradeprovision.alpha13

fixup source4/scripting/bin/samba_upgradedns
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1a4e95da by Noel Power at 2018-10-23T03:50:27Z
s4/scripting/bin: PY3 make sure GUID result of format is string

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c377bcd4 by Noel Power at 2018-10-23T03:50:28Z
s4/scripting/bin: PY3 fix samba4.blackbox.upgradeprovision.alpha13

- - - - -
60e7aa86 by Noel Power at 2018-10-23T03:50:28Z
s4/scripting/bin: blobs needs to be bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b142e28e by Noel Power at 2018-10-23T03:50:28Z
python/samba: PY3 fix samba4.blackbox.upgradeprovision.release-4-0-0

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
55f51476 by Noel Power at 2018-10-23T03:50:28Z
selftest/filter: PY3 Make filter-subunit forgiving of decoding errors

samba4.local.ndr for one is a test that outputs string in an encoding
that stdin.readline() guesses to be utf8 (but it isn't) filter subunit
can afford to be forgiving of some random text that can't be decoded as
utf8 so lets do that.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7562a551 by Noel Power at 2018-10-23T03:50:28Z
python/samba/netcmd: PY3 Fix error in samba4.blackbox.schemaupgrade

Getting Exception: must be str, not ldb.bytes error in scheme_upgrade
phase of test
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
09f6b864 by Noel Power at 2018-10-23T03:50:28Z
python/samba: PY3 port samba4.blackbox.functionalprep

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7aa40311 by Noel Power at 2018-10-23T03:50:28Z
python/samba/netcmd: PY3 fix samba.tests.domain_backup_offline

Fix attributes that need to be treated as str not bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a114ff21 by Noel Power at 2018-10-23T03:50:29Z
python/samba/samba3: PY3 tdb.Tdb.get method expects bytes in PY3

part of PY3 port samba4.blackbox.upgrade.samba3-upgrade*

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2c0f7f07 by Noel Power at 2018-10-23T03:50:29Z
python/samba/netcmd: PY3 make sure get_testparm_var returns 'str'

part of PY3 port samba4.blackbox.upgrade.samba3-upgrade*

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
86e22c06 by Noel Power at 2018-10-23T03:50:29Z
python/samba/provision: PY3 port samba4.blackbox.provision-backend

Fix some attibrutes that need to be treated as str

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cf38a067 by Noel Power at 2018-10-23T03:50:29Z
python/samba: PY3 port samba4.blackbox.provision-backend

convert ldif content to str when necessary

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c11e90a1 by Noel Power at 2018-10-23T03:50:29Z
python/samba/provision: PY3 PY3 port samba4.blackbox.provision-backend

Enclose filter with list as filter object has no 'len' method

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
25b16fa0 by Noel Power at 2018-10-23T03:50:29Z
python/samba/netcmd: PY3 fix CI error for samba.tests.samba_tool.help

Strangely the test was failing on CI only, looks like there is an
issue with order of elements returned from dict.items() with python3.4
(version of python in CI docker instance) and python3.6 (version on my
development machine). Changed code to sort the keys so order of help
printed out should be the same for each invocation.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f8b5dd95 by Noel Power at 2018-10-23T03:50:29Z
script: Add new (temporary) autobuild task for none-env tests.

Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8ba6bb59 by Noel Power at 2018-10-23T03:50:29Z
Add new CI job for new purepy3-none-env autobuild task

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d3571415 by Noel Power at 2018-10-23T07:10:22Z
CI/Autobuild: Remove samba-none-env-py3 test

We now run a purepython3 none-env test, later when the whole
build is running under python3 we will resurrect this job
but as (samba-none-env-py2) for python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Oct 23 09:10:22 CEST 2018 on sn-devel-144

- - - - -
ec209d28 by Volker Lendecke at 2018-10-25T15:58:22Z
tdb: Avoid casts

We have %PRIu32 and %zu these days

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
c37bf2f9 by Volker Lendecke at 2018-10-25T15:58:23Z
tdb: Use explicit initialization

Let the compiler figure out the optimal code

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
a895cc2a by Volker Lendecke at 2018-10-25T15:58:23Z
tdb: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
8df11518 by Volker Lendecke at 2018-10-25T15:58:23Z
tdb: Remove "USE_RIGHT_MERGES" code

This has not been activated by default for ages and can be very
inefficient. With check_merge_with_left_record() we have an
alternative that will merge freelist records while we walk it
anyway. This has reduced fragmentation significantly

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
afc616b6 by Volker Lendecke at 2018-10-25T15:58:24Z
tdbtorture: No transaction with mutexes

Right now we don't do transactions with mutexed tdbs. tdbtorture -m
locks up. I haven't really investigated why that is the case. The lockup
confused me quite a bit until I figured out it works fine as long as it
does not do transactions, which is all we need right now.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
e2d9abd4 by Volker Lendecke at 2018-10-25T15:58:24Z
tdbtorture: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
34b0f41c by Volker Lendecke at 2018-10-25T15:58:24Z
smbd: Replace some GUID_string by GUID_buf_string

It's only debug statements, but I would like to promote the
stack-allocation routines as good practice where they make sense.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
74de5a5d by Volker Lendecke at 2018-10-25T15:58:24Z
lib: Avoid an "includes.h"

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
05954fce by Volker Lendecke at 2018-10-25T15:58:24Z
smbd: Slightly optimize delay_rename_for_lease_break

Do the checks with increasing cost, possibly avoid more expensive ones

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
a03804c4 by Volker Lendecke at 2018-10-25T19:44:17Z
s3:smbd: Move a variable declaration closer to its use

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Thu Oct 25 21:44:17 CEST 2018 on sn-devel-144

- - - - -
b3e913ed by Douglas Bagnall at 2018-10-25T19:45:52Z
python/tests/unix: fix spelling and import of text_type

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
3f56dc1a by Douglas Bagnall at 2018-10-25T19:45:52Z
python/tests/gpo: import what we need (errno, not gp_log)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
2ce98901 by Douglas Bagnall at 2018-10-25T19:45:52Z
python/tests/__init__: import what we need

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
24a3acb2 by Douglas Bagnall at 2018-10-25T19:45:52Z
samba-tool drs: remove duplicate and unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
ca40183d by Douglas Bagnall at 2018-10-25T19:45:52Z
samba-tool ldapcmp: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
c3b53264 by Douglas Bagnall at 2018-10-25T19:45:52Z
samba-tool user: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
fe6daff2 by Douglas Bagnall at 2018-10-25T19:45:53Z
samba-tool domain: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
db6ab2cf by Douglas Bagnall at 2018-10-25T19:45:53Z
python/domain_update: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
daea7c1b by Douglas Bagnall at 2018-10-25T19:45:53Z
python/forest_update: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
74a5448b by Douglas Bagnall at 2018-10-25T19:45:53Z
python/gp_ext_loader: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
8519502c by Douglas Bagnall at 2018-10-25T19:45:53Z
python/ntacls: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
9d24ded2 by Douglas Bagnall at 2018-10-25T19:45:53Z
python/schema: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
1ecdde4f by Douglas Bagnall at 2018-10-25T19:45:53Z
python/gp_parse/gp_pol: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
f5df55d2 by Douglas Bagnall at 2018-10-25T19:45:53Z
python/provision: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
018c0138 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/dns*: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
0c18c373 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/dsdb_schema_attr: remove unused/duplicate imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
4fc99a04 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/getdcname: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
eac85232 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/netbios: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
b9161e52 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/*: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
135a3c29 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests/samdb: avoid useless local variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
68b284f6 by Douglas Bagnall at 2018-10-25T19:45:54Z
python/tests: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
5773290a by Douglas Bagnall at 2018-10-25T19:45:54Z
script/generate_param.py: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
dd99bfd8 by Douglas Bagnall at 2018-10-25T19:45:55Z
wintest/test-s3: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
6b88685f by Douglas Bagnall at 2018-10-25T19:45:55Z
ldb-samba/tests/match-rules: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
eef11c44 by Douglas Bagnall at 2018-10-25T19:45:55Z
ldb/tests/_ldb_text: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
02ed2599 by Douglas Bagnall at 2018-10-25T19:45:55Z
talloc/test/pytalloc: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
87272cf1 by Douglas Bagnall at 2018-10-25T19:45:55Z
tdb/test/_tdbtext: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
cb2c5843 by Douglas Bagnall at 2018-10-25T19:45:55Z
tests/blackbox/py: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
ff1bff9e by Douglas Bagnall at 2018-10-25T19:45:55Z
py/tests: remove unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
742c786d by Douglas Bagnall at 2018-10-25T19:45:55Z
s4/dsdb/pytest/dirsync: do not double import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
682b223d by Douglas Bagnall at 2018-10-25T19:45:55Z
s4/dsdb/pytest/dsdb_schema_info: do not double import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
86aca4db by Douglas Bagnall at 2018-10-25T19:45:56Z
s4/dsdb/pytest/: unused imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
3d5ba7fc by Douglas Bagnall at 2018-10-25T19:45:56Z
auth/cred/tests/bind: remove unused import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
da9fe70d by Douglas Bagnall at 2018-10-25T19:45:56Z
samba-tool: samba.getopt is not used here

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
e9a57a5b by Douglas Bagnall at 2018-10-25T19:45:56Z
s4/scripting/pfm_verify: remove duplicate import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
b90b2fcd by Douglas Bagnall at 2018-10-25T19:45:56Z
python/tests/kcc.graph: avoid import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
bbf1740b by Douglas Bagnall at 2018-10-25T19:45:56Z
python/tests/kcc_*: avoid * imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
aff2afd4 by Douglas Bagnall at 2018-10-25T19:45:56Z
selftest/perftests: avoid import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
c9c3a9ce by Douglas Bagnall at 2018-10-25T19:45:56Z
python/tests/common: avoid import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
8818b21e by Douglas Bagnall at 2018-10-25T19:45:57Z
selftest/tests.py: avoid import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
9f7247d8 by Douglas Bagnall at 2018-10-25T19:45:57Z
s3/selftest/tests: don't use import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
91bab6f8 by Douglas Bagnall at 2018-10-25T19:45:57Z
s4/selftest/tests: don't use import *

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
a1450f2c by Douglas Bagnall at 2018-10-25T19:45:57Z
pytest/dcerpc.integer: force py2 long int without incompatible syntax

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
2412ca51 by Douglas Bagnall at 2018-10-25T19:45:57Z
python tests: always use Python's unicodedata

We had our own special version with very few entries, but only
used it in one place.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
bc8201bd by Douglas Bagnall at 2018-10-25T19:45:57Z
samba-tool ldapcmp: use ValueError, not obsolete StandardError

The error is in the value, and StandardError is not in Python 3

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
71825bc9 by Douglas Bagnall at 2018-10-25T19:45:57Z
python/samba/common: py3 compat raw_input

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
d6072cbf by Douglas Bagnall at 2018-10-25T19:45:57Z
python/remove_dc: use a local variable in offline_remove_server

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
159fdcb6 by Douglas Bagnall at 2018-10-25T19:45:58Z
python/kcc: use compat.cmp_fn (PY3)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
cd5ac176 by Douglas Bagnall at 2018-10-25T19:45:58Z
script/show_test_time: approach python 3 compatibility

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
41844d13 by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/dsdb/pytests: Py3 compatitble except clauses

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
d121c68a by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/dsdb/pytest/ad_dc_medley: do not use xrange

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
4fdccc0d by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/dsdb/pytest/sort: use compat.cmp_fn instead of cmp

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
6ab7a3be by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/scripting: py3 style 0o123 octal, not 0123

this works with py2.6+ too.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
105e14cb by Douglas Bagnall at 2018-10-25T19:45:58Z
s4/scripting/demodirsync: fix syntax error

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
bab4f3a1 by Douglas Bagnall at 2018-10-25T19:45:58Z
python/gp_parse/gp_inf: remove shadowed method

The 'from_xml()' definition is replaced a few lines down

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
f12fd900 by Douglas Bagnall at 2018-10-25T19:45:58Z
python/gp_parse/gp_inf: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
03f45656 by Douglas Bagnall at 2018-10-25T19:45:59Z
python/tests/lsa_string: remove duplicate method

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
d042a300 by Douglas Bagnall at 2018-10-25T19:45:59Z
python/tests/kcc_utils: disambiguate/unshadow a test

Pair-programmed-with: Garming Sam <garming at catalyst.net.nz>
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
d2fadfd6 by Douglas Bagnall at 2018-10-25T19:45:59Z
ldb/tests/py/api: reveal shadowed casefold test

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
31172794 by Douglas Bagnall at 2018-10-25T19:45:59Z
s4/dsdb/pytest/ldap_schema: remove duplicate test

The tests differ in two lines, thus:

     def test_subClassOf(self):
     -        """ Testing usage of custom child schamaClass
     +        """ Testing usage of custom child classSchema

[...]

   -governsId: 1.3.6.1.4.1.7165.4.6.2.6.3.""" + str(random.randint(1, 100000)) + """
   +governsId: 1.3.6.1.4.1.7165.4.6.2.6.7.""" + str(random.randint(1, 100000)) + """

with the governsId OID changed because it was noticed they were colliding.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
99be8e98 by Douglas Bagnall at 2018-10-25T19:45:59Z
python/tests/raw_protocol: reveal shadowed test via disambiguation

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
add4068f by Douglas Bagnall at 2018-10-25T19:45:59Z
samba-tool tests: fix bytes/str issue in masked test

This test will be revealed to the world in the next commit.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
654cc08c by Douglas Bagnall at 2018-10-25T19:45:59Z
python/tests/ou: unshadow a test

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
adf73d34 by Douglas Bagnall at 2018-10-25T22:50:37Z
pytests/samba3sam: unshadow and fix a search_non_mapped test

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Fri Oct 26 00:50:37 CEST 2018 on sn-devel-144

- - - - -
525b19fa by Christian Ambach at 2018-10-26T04:59:08Z
s3:script/tests reduce code duplication

Signed-off-by: Christian Ambach <ambi at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b89732c3 by Christian Ambach at 2018-10-26T04:59:09Z
s3:utils/smbget add error handling for mkdir() calls

Signed-off-by: Christian Ambach <ambi at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
fce0d1b2 by Christian Ambach at 2018-10-26T07:58:07Z
s3:utils/smbget fix recursive download with empty source directories

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13199
Signed-off-by: Christian Ambach <ambi at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Oct 26 09:58:07 CEST 2018 on sn-devel-144

- - - - -
ec4b2ac5 by Andreas Schneider at 2018-10-27T19:24:23Z
s3:selftest: Fix test names of smbtorture_s3.plain

The env name will be appended. There is no need to have it twice. Can't
we remove the tests againa ad_dc_ntvfs completely?

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4db918b4 by Andreas Schneider at 2018-10-27T19:24:23Z
s3:torture: Don't use the same testdir twice

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d48a2bc3 by Andreas Schneider at 2018-10-27T19:24:23Z
s3:torture: Rename the test file and remove it if it exists

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
94f8a690 by Volker Lendecke at 2018-10-27T19:24:24Z
lib: Remove unused tdb_trans_* functions

The transactions have all moved to dbwrap

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5391e21d by David Mulder at 2018-10-27T22:35:34Z
lib:socket: If returning early, set ifaces

Prevents a segfault in load_interfaces() when total interfaces == 1.
Fixes regression caused by da68a1b2f417ec82ea4ed3e7a4d867cef8ca8f93.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13665

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Ralph Böhme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Sun Oct 28 00:35:35 CEST 2018 on sn-devel-144

- - - - -
ba17cae4 by Andreas Schneider at 2018-10-29T19:09:25Z
s3:winbind: Check return code of initialize_password_db()

See https://retrace.fedoraproject.org/faf/reports/1577174/

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13668

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0b60afec by Douglas Bagnall at 2018-10-29T19:09:25Z
python/samdb: properly use property()

Python's property() function works like this:

property([getter[, setter[, delete[, doc]]]])

but we have been forgetting the delete function, or rather setting it
to be a string. A string is not callable and is unlikely to succeed at
deleting the property.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b37f8f88 by Douglas Bagnall at 2018-10-29T22:13:36Z
python: do not use "is" for string equality

This is not always going to work, and is not guaranteed to be
consistent even between minor versions.

Here is a simple counterexample:

>>> a = 'hello'
>>> a is 'hello'
True
>>> a is 'hello'.lower()
False
>>> a == a.lower()
True

Possibly it always works for the empty string, but we cannot rely
on that.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Mon Oct 29 23:13:36 CET 2018 on sn-devel-144

- - - - -
5d565f63 by Ralph Boehme at 2018-10-29T22:36:24Z
vfs_fruit: remove check for number of xattrs from ad_convert_xattr

Turns out that there exist AppleDouble files with an extended FinderInfo
entry that includes the xattr marshall buffer, but the count of xattrs
in the buffer is just zero.

We do want to discard this extended FinderInfo entry and convert it to a
simple fixed size FinderInfo entry, so remove the check.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13649

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
14abead3 by Volker Lendecke at 2018-10-29T22:36:24Z
tdb: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c18e2ed0 by Volker Lendecke at 2018-10-29T22:36:24Z
tdb: Align an integer type

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
96f6768e by Volker Lendecke at 2018-10-29T22:36:24Z
tdb: Don't delete dead records in traverse

The next commit will change the handling of dead records, removing the
"tdb_do_delete" function. As traverses should not happen in normal
operations, dead records from them should be rare, and relying on
traverses to remove them is a very bad idea IMHO.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4ed2a67a by Volker Lendecke at 2018-10-29T22:36:25Z
tdb: Purge dead records whenever we block the freelist

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
05212658 by Volker Lendecke at 2018-10-29T22:36:25Z
tdb: Do early RDONLY error check for tdb_delete

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
500a729a by Volker Lendecke at 2018-10-30T01:48:38Z
tdb: Make record deletion circular-chain safe

Before this patch we had 3 loops walking a hash chain to delete
records:

tdb_do_delete() to find the predecessor of the record that was to be
deleted. tdb_count_dead(), the name says it all and tdb_purge_dead()
to give back all dead records from a chain to the freelist.

This patch introduces tdb_trim_dead that walks a hash chain just
once. While it does so it counts the number of dead records, and all
records beyond tdb->max_dead_records are moved to the freelist.

Normal record deletion now works by always marking a record as dead in
step 1 and then calling tdb_trim_dead. This is made safe against
circular chains by doing the slow chain walk only in the case when we
did not delete a dead record during our walk.

It changes our dynamics a bit:

When deleting a record with non-zero max_dead_records, now we always
leave that number of records around when deleting, doing a blocking
lock on the freelist when we found too many dead records.

Previously when exceeding max_dead_records we wiped all dead records
to start accumulating them from scratch, assuming we could lock the
freelist in a nonblocking fashion.

The net effect for an uncontended freelist is the same: In
tdb_allocate() we still completely hand over all dead records to the
freelist when we could lock it, it just happens later than without
this patch.

This means for a lightly loaded system we will potentially leave more
dead records around in databases like locking.tdb. However, on a
heavily loaded system we become more predictable: If the freelist is
so heavily contended that across many deletes we can't get hold of it,
previously we accumulated more dead records than max_dead_records
would allow. This is a really lowlevel tradeoff that is likely hard to
measure, but to me becoming more deterministic without sacrificing too
much parallelism (we keep more dead records around) is worth trying.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Oct 30 02:48:38 CET 2018 on sn-devel-144

- - - - -
0189f23f by Stefan Metzmacher at 2018-10-30T06:30:20Z
schema_samba4.ldif: add allocation of DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME

This was already allocated in source4/dsdb/samdb/samdb.h with
commit 22208f52e6096fbe9413b8ff339d9446851e0874.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
60131b44 by Stefan Metzmacher at 2018-10-30T06:30:20Z
s4:dsdb: fix comment on DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
364ed537 by Stefan Metzmacher at 2018-10-30T06:30:20Z
testprogs/blackbox: add samba4.blackbox.test_primary_group test

This demonstrates the bug, that happens when the primaryGroupID
of a user is changed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bb9c9e49 by Stefan Metzmacher at 2018-10-30T06:30:20Z
s4:dsdb: add DSDB_CONTROL_DBCHECK_FIX_LINK_DN_SID oid

This will be used to fix missing <SID=> components in future.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c5c99b56 by Stefan Metzmacher at 2018-10-30T06:30:20Z
dbchecker: improve verbose output of do_modify()

This makes it easier to debug dbcheck problems.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a801799e by Stefan Metzmacher at 2018-10-30T06:30:20Z
dbchecker: Fix missing <SID=...> on linked attributes

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f81771c8 by Stefan Metzmacher at 2018-10-30T06:30:20Z
blackbox/dbcheck-links: Test broken links with missing <SID=...> on linked attributes

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
966c7feb by Stefan Metzmacher at 2018-10-30T06:30:20Z
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_handle_linked_attribs()

This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
42e69a86 by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_add()

This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
70a306d0 by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: add missing \n to a DEBUG message in replmd_modify_la_add()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
738b52eb by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_delete()

This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1ef145d9 by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: pass down struct replmd_replicated_request to replmd_modify_la_replace()

This will simplify further changes.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0386307e by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:repl_meta_data: add support for DSDB_CONTROL_DBCHECK_FIX_LINK_DN_SID

This will be used by dbcheck in the next commits.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7a36cb30 by Stefan Metzmacher at 2018-10-30T06:30:21Z
s4:samldb: internally use extended dns while changing the primaryGroupID field

This is important, otherwise we'll loose the <SID=> component of the
linked attribute.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
852e1db1 by Andrew Bartlett at 2018-10-30T09:32:51Z
dsdb: Add comments explaining the limitations of our current backlink behaviour

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13418

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Tim Beale <timbeale at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Oct 30 10:32:51 CET 2018 on sn-devel-144

- - - - -
eeb4089d by Gary Lockyer at 2018-10-30T15:40:13Z
dsdb group_audit: Test to replicate BUG 13664

The group audit code incorrectly logs member additions and deletions.

Thanks to metze for the debugging that isolated the issue, and for
suggesting the fix to dn_compare.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8420a4d0 by Gary Lockyer at 2018-10-30T15:40:13Z
dsdb group audit: align dn_compare with memcmp

Rename the parameter names and adjust the  return codes from dn_compare
so that:
dn_compare(a, b) =>

LESS_THAN means a is less than b.
GREATER_THAN means a is greater than b.

Thanks to metze for suggesting the correct semantics for dn_compare

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13664

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
e2970887 by Gary Lockyer at 2018-10-30T15:40:13Z
dsdb group audit tests: check_timestamp improve diagnostics

Change check_timestamp to display the expected, actual along with the
line and name of the failing test, rather than the line in
check_timestamp.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
c952fc12 by Gary Lockyer at 2018-10-30T15:40:13Z
dsdb group audit tests: check_version improve diagnostics

Change check_version to display the expected, actual along with the
line and name of the failing test, rather than the line in check_version

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
aeef8b41 by Gary Lockyer at 2018-10-30T19:20:26Z
dsdb group audit tests: log_membership_changes extra tests

Add extra tests to ensure better test coverage of log_membership_changes

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Tue Oct 30 20:20:26 CET 2018 on sn-devel-144

- - - - -
b40daca6 by Tim Beale at 2018-10-30T23:30:16Z
traffic_replay: Generate users faster by writing to local DB

We can create user accounts much faster if the LDB connection talks
directly to the local sam.ldb file rather than going via LDAP. This
patch allows the 'host' argument to the tool to be a .ldb file (e.g.
"/usr/local/samba/private/sam.ldb") instead of a server name/IP.

In most cases, the traffic_replay tool wants to run on a remote device
(because the point of it is to send traffic to the DC). However, the
--generate-users-only is one case where the tool can be run locally,
directly on the test DC. (The traffic_replay user generation is handy
for standalone testing, because it also handles assigning group
memberships to the generated user accounts).

Note that you also need to use '--option="ldb:nosync = true"' to get
the improvement in performance.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
49434731 by Tim Beale at 2018-10-30T23:30:16Z
traffic_replay: Change print() to use logger()

This reduces noise, so the messages only come out if you specify
--debug.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
4a8a0ab3 by Tim Beale at 2018-10-30T23:30:16Z
traffic_replay: logger was ignoring smb.conf log-level

We were trying to access the debug-level (in python C bindings) before
the smb.conf had been loaded and actually set the debug-level. So it
would default to zero, regardless of what was in the smb.conf.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5a57378d by Tim Beale at 2018-10-30T23:30:16Z
netcmd: Add backend-store option to domain backup/rename cmds

Currently the online/rename backup files always use the default backend
(TDB) and there is no way to change this.

This patch adds the backend-store option to the backup commands so that
you can create a backup with an MDB backend, if needed.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a9a65adb by Tim Beale at 2018-10-30T23:30:16Z
selftest: Specify different DB backends for restored testenvs

Vary the DB backend that we use for the renamed DCs. The labdc and
renamedc are fairly similar, so let's have each of them use a different
backend.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
ca570bd4 by Tim Beale at 2018-10-30T23:30:16Z
netcmd: Include num-members in 'samba-tool group list --verbose'

This adds an easy way for users to see (via samba-tool) how many members
are in various groups, without querying the members for each individual
group.

For example, you could pipe this output to grep to check for groups with
zero or one members (i.e. historic groups that may no longer make
sense).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
0c910245 by Tim Beale at 2018-10-31T02:40:41Z
netcmd: Add 'samba-tool group stats' command

With large domains it's hard to get an idea of how many groups there
are, and how many users are in each group, on average. However, this
could have a big impact on whether a problem can be reproduced or not.

This patch dumps out some summary information so that you can get a
quick idea of how big the groups are.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Wed Oct 31 03:40:41 CET 2018 on sn-devel-144

- - - - -
bda98b38 by Andreas Schneider at 2018-10-31T20:27:16Z
third_party: Update nss_wrapper to version 1.1.5

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9a4c1888 by Ralph Boehme at 2018-10-31T20:27:16Z
s3:smbd: remove "0x" string prefix from dev/ino

We used %llu as conversion specifier which results in a decimal number
being printed, so remove the misleading "0x" prefix.

While at it, I'll change %llu to the terse %ju.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f7a98094 by Ralph Boehme at 2018-10-31T20:27:17Z
docs:vfs_fruit: add "wipe_intentionally_left_blank_rfork" option

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2a9574b1 by Ralph Boehme at 2018-10-31T20:27:17Z
docs:vfs_fruit: add "delete_empty_adfiles" option

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c3a1f3ec by Ralph Boehme at 2018-10-31T20:27:17Z
s3:selftest: list vfs testssuites one per line

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6f022e61 by Ralph Boehme at 2018-10-31T20:27:17Z
s4:torture: add test for AppleDouble ResourceFork conversion

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2dbb2d28 by Ralph Boehme at 2018-10-31T20:27:17Z
vfs_fruit: add option "wipe_intentionally_left_blank_rfork"

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7be979f9 by Ralph Boehme at 2018-10-31T20:27:17Z
vfs_fruit: detect empty resource forks in ad_convert()

For some reason the macOS client often writes AppleDouble files with a
non-zero sized resource fork, but the resource fork data is just
boilerplate data with the following string close to the start

  This resource fork intentionally left blank

A dump with apple_dump looks like this:

Entry ID   : 00000002 : Resource Fork
Offset     : 00000052 : 82
Length     : 0000011E : 286

-RAW DUMP--:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F : (ASCII)
00000000   : 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 1E : ................
00000010   : 54 68 69 73 20 72 65 73 6F 75 72 63 65 20 66 6F : This resource fo
00000020   : 72 6B 20 69 6E 74 65 6E 74 69 6F 6E 61 6C 6C 79 : rk intentionally
00000030   : 20 6C 65 66 74 20 62 6C 61 6E 6B 20 20 20 00 00 :  left blank   ..
00000040   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000050   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000060   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000070   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000080   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000090   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000A0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000B0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000C0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000D0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000E0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
000000F0   : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
00000100   : 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 1E : ................
00000110   : 00 00 00 00 00 00 00 00 00 1C 00 1E FF FF       : ..............

We can safely discard this Resource Fork data.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e00e6134 by Ralph Boehme at 2018-10-31T20:27:17Z
vfs_fruit: add option "delete_empty_adfiles"

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3649f1a4 by Ralph Boehme at 2018-10-31T20:27:18Z
vfs_fruit: optionally delete AppleDouble files without Resourcefork data

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13642

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9f58997d by Ralph Boehme at 2018-10-31T20:27:18Z
vfs_streams_xattr: fix open implementation

Since a long time the modules's open function happily returned success
when opening a non existent stream without O_CREAT.

This change fixes it to return -1 and errno=ENOATTR if

o get_ea_value() returns NT_STATUS_NOT_FOUND (eg mapped from
  getxattr() = -1, errno=ENOATTR) and

o flags doesn't contain O_CREAT

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
242f42ef by Ralph Boehme at 2018-10-31T20:27:18Z
s4:torture/vfs/fruit: skip a few tests when running against a macOS SMB server

These tests are designed to test specific vfs_fruit functionality.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
394d1fc9 by Ralph Boehme at 2018-10-31T20:27:18Z
s4:torture/vfs/fruit: fix a few error checks in "delete AFP_AfpInfo by writing all 0"

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1d4e5064 by Ralph Boehme at 2018-10-31T20:27:18Z
s4:torture/vfs/fruit: set share_access to NTCREATEX_SHARE_ACCESS_MASK in check_stream_list

Avoid sharing conflicts with other opens on the basefile.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dba9a474 by Ralph Boehme at 2018-10-31T20:27:18Z
s4:torture/vfs/fruit: update test "SMB2/CREATE context AAPL" to work against macOS

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
12549886 by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: update test "stream names" to work with macOS

o create the basefile before trying to create a stream on it, otherwise
  this fails on macOS

o write something to the stream, otherwise the stream is not listed as
  macOS hides 0-byte sized streams

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f7551d8f by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: ensure a directory handle is closed in all code paths

Otherwise we get a sharing violation when running against Samba and
opening the directory a second time.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9cd9859d by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: update test "read open rsrc after rename" to work with macOS

macOS SMB server seems to return NT_STATUS_SHARING_VIOLATION in this
case while Windows 2016 returns NT_STATUS_ACCESS_DENIED.

Lets stick with the Windows error code for now in the Samba fileserver,
but let the test pass against macOS.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0d9a80b6 by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: expand existing test "setinfo delete-on-close AFP_AfpInfo" a little bit

Add a check that verifies a create on a stream gets
NT_STATUS_DELETE_PENDING after delete-on-close has been set.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6f428607 by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: expand existing vfs_test "null afpinfo"

This adds a check that a read on a seperate handle also sees the
previously created AFP_AfpInfo stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c3cf09a0 by Ralph Boehme at 2018-10-31T20:27:19Z
s4:torture/vfs/fruit: update test "creating rsrc with read-only access" for newer macOS versions

While this operation failed against older macOS versions, it passes
against versions 10.12 and newer. Update the test accordingly, a
subsequent commit will then update our implementation.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
055ca448 by Ralph Boehme at 2018-10-31T20:27:19Z
vfs_fruit: update handling of read-only creation of resource fork

macOS SMB server versions supports this since 10.12, so we adapt our
behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b35d9e74 by Ralph Boehme at 2018-10-31T20:27:20Z
s4:torture/vfs/fruit: expand test "setinfo eof stream"

o Adds checks verifying that after setting eof to 0 on a stream, a
  subsequent open gets ENOENT, before and after closing the handle that
  had been used to set eof to 0.

o Verify that a write to a handle succeeds after that handle has been
  used to set eof to 0 on a stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3195ff95 by Ralph Boehme at 2018-10-31T20:27:20Z
s4:torture/vfs/fruit: write some data to a just created teststream

Doesn't currently make a difference, but this prepares for a later
change in vfs_fruit that will filter out empty streams (which is the
macOS behaviour).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
287082d7 by Ralph Boehme at 2018-10-31T20:27:20Z
vfs_fruit: don't unlink 0-byte size truncated streams

This caused all sort of havoc with subsequent SMB request that acted on
the handle of the then deleted backend storage (file or blob, depending
on the used streams module).

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7485007b by Ralph Boehme at 2018-10-31T20:27:20Z
s4:torture/vfs/fruit: enable AAPL extensions in a bunch of tests

These tests check for macOS SMB server specific behaviour. They work
currently against Samba without enabling AAPL because in vfs_fruit we're
currently don't check whether AAPL has been negotiated in one place. A
subsequent commit will change that and this commit prepares for that
change.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
046456fc by Ralph Boehme at 2018-10-31T20:27:20Z
vfs_fruit: use check on global_fruit_config.nego_aapl for macOS specific behaviour

Ensure any non MS compliant protocol behaviour targetted at supporting
macOS clients are only effective if the client negotiated AAPL.

Currently this only guards the resource fork which only macOS client are
going to use, but subsequent commits add more this at this place.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ebfcf75e by Ralph Boehme at 2018-10-31T20:27:20Z
vfs_fruit: filter empty streams

First step in achieving macOS compliant behaviour wrt to empty streams:
- hide empty streams in streaminfo
- prevent opens of empty streams

This means that we may carry 0-byte sized streams in our streams
backend, but this shouldn't really hurt.

The previous attempt of deleting the streams when an SMB setinfo eof to
0 request came in, turned out be a road into desaster.

We could set delete-on-close on the stream, but that means we'd have to
check for it for every write on a stream and checking the
delete-on-close bits requires fetching the locking.tdb record, so this
is expensive and I'd like to avoid that overhead.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
64b28e4a by Ralph Boehme at 2018-10-31T20:27:20Z
s4:torture/util: add torture_smb2_open()

This seems to be missing: a simple wrapper to just open a file without
fancy options.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
aba502d4 by Ralph Boehme at 2018-10-31T20:27:21Z
s4:torture/vfs/fruit: add check_stream_list_handle()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9d8751db by Ralph Boehme at 2018-10-31T20:27:21Z
s4:torture/vfs/fruit: add test "empty_stream"

One to rule them all: consistently test critical operations on all
streams relevant to macOS clients: the FinderInfo stream, the Resource
Fork stream and an arbitrary stream that macOS maps to xattrs when
written to on a macOS SMB server.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
993c7c4e by Ralph Boehme at 2018-10-31T20:27:21Z
vfs_fruit: add some debugging of dev/ino

Aids in debugging dev/ino mismatch failures in open_file_ntcreate.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a26032c3 by Ralph Boehme at 2018-10-31T20:27:21Z
vfs_fruit: remove resource fork special casing

Directly unlinking a file with open handles is not good, don't do it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1e055a79 by Ralph Boehme at 2018-10-31T20:27:21Z
vfs_fruit: add fio->created

fio->created tracks whether a create created a stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
96320ecc by Ralph Boehme at 2018-10-31T20:27:21Z
vfs_fruit: prepare struct fio for fake-fd and on-demand opening

Not used for now, that comes in the subsequent commits.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4a5c9a9e by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: prepare fruit_pwrite_meta() for on-demand opening and writing

This avoid creating files or blobs in our streams backend when a client
creates a stream but hasn't written anything yet. This is the only sane
way to implement the following semantics:

* client 1: create stream "file:foo"

* client 2: open stream "file:foo"

The second operation of client 2 must fail with NT_STATUS_NOT_FOUND.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d7d92710 by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: prepare fruit_pread_meta() for reading on fake-fd

If the read on the stream fails we may have hit a handle on a just
created stream (fio->created=true) with no data written yet.

If that's the case return an empty initialized FinderInfo blob.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
80afafe3 by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: do ino calculation

As we'll start returning fake fds in open shortly, we can't rely on the
next module to calculat correct inode numbers for streams and must take
over that responsibility.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6fd256af by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: let fruit handle all aio on the FinderInfo metadata stream

This will be required to support using fake fds for the FinderInfo
metadata stream.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6e13dbdd by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: pass stream size to delete_invalid_meta_stream()

delete_invalid_meta_stream() is meant to guard against random data being
present in the FinderInfo stream. If the stream size is 0, it's likely a
freshly created stream where no data has been written to yet, so don't
delete it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
480695cd by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: let fruit_pwrite_meta_stream also ftruncate empty FinderInfo

fruit_streaminfo currently filters out the FinderInfo stream is
delete-on-close is set. We set it here internally, but the client may
also set it over SMB. Turns out that the macOS SMB server does NOT
filter out FinderInfo stream with delete-on-close set, so we must change
the way filtering is done in fruit_streaminfo.

Filtering is now done based on the FinderInfo stream being 0-bytes large which
is why I'm adding the ftruncate here.

No idea why the tests that check the filtering passed the commits
leading up to this one, but if you revert this commit after applying the
whole patchset, the "delete AFP_AfpInfo by writing all 0" test will fail.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d8c1bb52 by Ralph Boehme at 2018-10-31T20:27:22Z
vfs_fruit: don't check for delete-on-close on the FinderInfo stream

macOS SMB server doesn't filter out the FinderInfo stream if it has
delete-on-close set.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1b2de44e by Ralph Boehme at 2018-11-01T00:14:23Z
vfs_fruit: let fruit_open_meta() with O_CREAT return a fake-fd

This is the final step in implementing the needed macOS semantics on the
FinderInfo stream: as long as the client hasn't written a non-zero
FinderInfo blob to the stream, there mustn't be a visible filesystem
entry for other openers.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13646

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Nov  1 01:14:23 CET 2018 on sn-devel-144

- - - - -
8d14714c by Samuel Cabrero at 2018-11-01T00:59:10Z
s3: winbind: Remove fstring from wb_acct_info struct

The group enumeration backend functions try to allocate an array of
wb_acct_info structs with a number of elements equal to the number of
groups. In domains with a large number of groups this allocation may
fail due to the size of the chunk.

Found while trying to enumerate the groups in a domain with more than
700k groups.

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2cfb58d7 by Ralph Wuerthner at 2018-11-01T00:59:10Z
nsswitch: use goto to have only one function return

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e82b3ac0 by Ralph Wuerthner at 2018-11-01T00:59:10Z
nsswitch: make wb_global_ctx private add add get/put functions to access global context

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
988182c3 by Ralph Wuerthner at 2018-11-01T00:59:10Z
nsswitch: protect access to wb_global_ctx by a mutex

This change will make libwbclient thread safe for all API calls not using a
context. Especially there are no more conflicts with threads using nsswitch
and libwbclient in parallel.

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b5ea7946 by Ralph Wuerthner at 2018-11-01T00:59:10Z
nsswitch: add test for parallel NSS & libwbclient calls

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
40bd0a93 by Volker Lendecke at 2018-11-01T04:06:23Z
nsswitch: Run nsswitch thread test

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Nov  1 05:06:23 CET 2018 on sn-devel-144

- - - - -
371297a5 by Douglas Bagnall at 2018-11-01T04:08:09Z
tests/samba_tool/provision_password_check: follow super inheritance

We were skipping a level in the inheritance chain, which had no effect
in this case (no .setUps or .tearDowns were missed) but it would be
confusing if the parents ever changed.

Note: in python 3, you just call super() with no args, and it works
out the right thing.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
99eee0a6 by Douglas Bagnall at 2018-11-01T04:08:09Z
selftest/format_subunit_json: py3 print()

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
a6368ae6 by Douglas Bagnall at 2018-11-01T04:08:09Z
s4/dsdb/pytest/ldap_schema: fix typo in docs

Commit 311727947799e896e05d644103c9db80a665de88 removed a duplicate
test but it removed the wrong one, leaving this dreadful typo.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
4e694079 by Douglas Bagnall at 2018-11-01T04:08:09Z
tests/python/notification: safer use of super()

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
6c1d6053 by Douglas Bagnall at 2018-11-01T04:08:09Z
tests/python/sec_descriptor: safer use of super()

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
85323088 by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/script/depfilter.py: use py3 compatible regex import

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
f109fde3 by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/scripting/autoidl: py3 compatible except

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
2e6fccbb by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/scripting/minschema: whitespace reformat and py3 compatible print

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
b4fff0ac by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/scripting/samba_upgradedns: avoid .has_key()

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
b1ba193d by Douglas Bagnall at 2018-11-01T04:08:10Z
s4/scripting/pfm_verify: PY3: use compat.text_type

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
2013cd28 by Douglas Bagnall at 2018-11-01T04:08:10Z
script/show_test_time: attempt py3 compat

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
2fedb950 by Douglas Bagnall at 2018-11-01T04:08:10Z
python: PY3 Exceptions don't have .message

but str(e) is the same as str(e.message), so we can use that
on 2 and 3.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
9057b1c4 by Douglas Bagnall at 2018-11-01T04:08:10Z
dbcheck: fix message formatting

previously these would have raised an exception

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
e621472c by Douglas Bagnall at 2018-11-01T04:08:10Z
dbcheck: fix function call (right arguments)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
e25878ec by Douglas Bagnall at 2018-11-01T04:08:11Z
samba-tool: let self.usage() find argv[0] by itself

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
ced6b6c9 by Douglas Bagnall at 2018-11-01T04:08:11Z
samba-tool user: fix message format

There were 2 % formats and 3 arguments.
Also reformat for line length

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
cf7f14a4 by Douglas Bagnall at 2018-11-01T04:08:11Z
python/subnets: use the correct variable name

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
5304ea44 by Douglas Bagnall at 2018-11-01T04:08:11Z
python/tests/dsdb: use correct variable names

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
f129ca98 by Douglas Bagnall at 2018-11-01T04:08:11Z
python/tests/dnscmd: don't use undefined name

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
fc64f258 by Douglas Bagnall at 2018-11-01T04:08:11Z
ldb/tests/py/api: use proper name for ldb.LdbError

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
78ead841 by Douglas Bagnall at 2018-11-01T04:08:11Z
ldb/tests/py/index: remove dup value in dictionary

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
9148ec7d by Douglas Bagnall at 2018-11-01T04:08:11Z
s4/dsdb/pytest/sites: do not use variables out of scope

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
4be0bcb1 by Douglas Bagnall at 2018-11-01T04:08:12Z
python join: use the sd_utils we imported

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
7058e21e by Douglas Bagnall at 2018-11-01T04:08:12Z
python: avoid useless work in dsdb_Dn.__cmp__

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
f0c45e08 by Douglas Bagnall at 2018-11-01T04:08:12Z
python dbcheck: use real exception name

and conventional indent

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
d9b45794 by Douglas Bagnall at 2018-11-01T04:08:12Z
Python provision: remove "ExistingBackend"

This was unused and broken. e.g. here:

-    def init(self):
-        # Check to see that this 'existing' LDAP backend in fact exists
-        ldapi_db = Ldb(self.ldapi_uri)

there is no attribute self.ldapi_uri, so this would always raise an
exception.

It was being left around in case it became useful, but that doesn't
seem to be happening.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
f6f98ed2 by Douglas Bagnall at 2018-11-01T04:08:12Z
provision: fix string format syntax

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
67c5ab17 by Douglas Bagnall at 2018-11-01T04:08:12Z
provision: fix string formatting (number of args)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
71f91ccd by Douglas Bagnall at 2018-11-01T04:08:12Z
provision/backend: fix formating syntax

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
7db224bd by Douglas Bagnall at 2018-11-01T04:08:12Z
provision/backend: LDAPBackendResult takes no creds

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
7fafd579 by Douglas Bagnall at 2018-11-01T04:08:13Z
python/remove_dc: fix formating syntax (missing %s)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
7b1d878f by Douglas Bagnall at 2018-11-01T04:08:13Z
python/remove_dc: fix Exception construction

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
d4e06603 by Douglas Bagnall at 2018-11-01T04:08:13Z
waflib: add necessary imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
aafe6bf9 by Douglas Bagnall at 2018-11-01T04:08:13Z
waflib: fix syntax error in string format

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
9c2447e2 by Douglas Bagnall at 2018-11-01T04:08:13Z
py/tests/dcerpc_rpc: Py3 compat integer types

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
9055602e by Douglas Bagnall at 2018-11-01T04:08:13Z
traffic_replay: Exception has no .message

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1a66eeb1 by Douglas Bagnall at 2018-11-01T04:08:13Z
tests/rodc_rwdc: fix message format

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1ca5bf32 by Douglas Bagnall at 2018-11-01T04:08:13Z
python/kcc/graph: import KCCError, which is used somewhere

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
68a43a18 by Douglas Bagnall at 2018-11-01T04:08:14Z
python/samba3: import passdb in the manner it is used

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
866d5b3c by Douglas Bagnall at 2018-11-01T04:08:14Z
samba-tool rodc: remove unused variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
202be599 by Douglas Bagnall at 2018-11-01T04:08:14Z
samba-tool spn: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
d5f1ea60 by Douglas Bagnall at 2018-11-01T04:08:14Z
dbchecker: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d7f26be9 by Douglas Bagnall at 2018-11-01T04:08:14Z
samba-tool dbcheck: remove unused variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
55aaa9c8 by Douglas Bagnall at 2018-11-01T04:08:14Z
python/upgradehelpers: remove unused variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
78f0860c by Douglas Bagnall at 2018-11-01T04:08:14Z
python/xattr: remove unused variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
cf3d35c8 by Douglas Bagnall at 2018-11-01T04:08:14Z
python/tests/credentials: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
e37edb76 by Douglas Bagnall at 2018-11-01T04:08:14Z
python/tests/source: remove useless local variable

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
5e796308 by Douglas Bagnall at 2018-11-01T04:08:15Z
autobuild: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
4e19b69b by Douglas Bagnall at 2018-11-01T04:08:15Z
py/tests/dcerpc_integer: remove dup tests

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
77b9b5e8 by Douglas Bagnall at 2018-11-01T04:08:15Z
samba-tool domain: remove unused variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
00eaea56 by Douglas Bagnall at 2018-11-01T04:08:15Z
samba-tool processes: remove unused imports and variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
d1777971 by Douglas Bagnall at 2018-11-01T04:08:15Z
python/upgradehelpers: remove unused provision imports

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
eeb1e81e by Douglas Bagnall at 2018-11-01T04:08:15Z
python/join: remove unused imports and variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5f19bf63 by Douglas Bagnall at 2018-11-01T04:08:15Z
python/tests/docs: remove unused import, variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
790acef2 by Douglas Bagnall at 2018-11-01T08:40:02Z
samba-tool ntacl: remove unused imports and variables

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Thu Nov  1 09:40:02 CET 2018 on sn-devel-144

- - - - -
d29d2f2f by Douglas Bagnall at 2018-11-01T08:41:19Z
pytests: allow blackbox subcommands without a shell

When given a list, it will use the list directly as an argument list,
avoiding shell-expansion and the intermediatory process.

This removes shell expansion trouble, and saves the machine a little
bit of work.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1451b696 by Tim Beale at 2018-11-01T08:41:19Z
selftest: Add new customdc testenv that can load any backup-file

This adds a new testenv that can be used for sandpit/manual testing.
This testenv can be based off any backup-file that you like.

The main use case is large databases. Populating 1000s of users is
time-consuming (it can take hours to create a really large DB). Instead
of having to manually add users to the testenv every time you want to
try something, this allows you to populate the users just once, take a
backup/snapshot of the DB, and then spin up the backup multiple times.

In theory this testenv could be useful for other situations too, e.g.
dealing with a corrupted database, testing DB migration (e.g. 4.7 -->
4.8), or if (for some reason) you wanted to create a realistic
lab-domain within a testenv.

To run-up the testenv you need to specify a BACKUP_FILE environment
variable (the same way we specify the SELFTEST_TESTENV), e.g.
  BACKUP_FILE=/files/backup-10k-ad_dc.tar.bz2 \
    SELFTEST_TESTENV=customdc make testenv

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c35fb3a8 by Tim Beale at 2018-11-01T08:41:19Z
dnsupdate: Pass smb.conf through to samba-tool commands

If you call samba_dnsupdate with a --configfile option, this wasn't
passed through to the samba-tool commands the script tries to run.
Normally, samba_dnsupdate would only be run on the DC itself, so it
shouldn't be a big deal, however, this may be a problem if you install
the samba database into a non-default location (i.e. not
/usr/local/samba).

This patch passes through the smb.conf file, if one was specified.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
018ff496 by Tim Beale at 2018-11-01T08:41:20Z
dnsupdate: Skip kerberos step if use-file specified

If there's a problem in get_credentials() (getting the machine account
Kerberos credentials), then we fallback to use_samba_tool (essentially
ignoring use-file). However, there's no need to do this, as use-file
shouldn't require Kerberos credentials.

This was making bootstrapping issues starting a testenv harder to debug.
Obviously, Kerberos is dependent on DNS functioning correctly, but
running dnsupdate was also dependent on having a working Kerberos KDC.
In my case, the testenv had a bad krb5.conf file, but the problem
appeared as resolv-wrapper errors (due to a missing RESOLV_WRAPPER_HOSTS
file, which should've been generated by dnsupdate).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Pair-Programmed-With: Garming Sam <garming at catalyst.net.nz>

- - - - -
e2ee5952 by Tim Beale at 2018-11-01T12:06:01Z
selftest: Remove unnecessary code for backup testenvs

setup_namespaces() already gets done for the backupfromdc's domain, so
this step is unnecessary for the restoredc and offlinebackupdc testenvs
(which are based off the backupfromdc's database).

The setup_namespaces() step is still necessary for the renamedc/labdc,
as these don't have the UPN/SPN suffixes for the new realm yet.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Nov  1 13:06:01 CET 2018 on sn-devel-144

- - - - -
9b28d47b by Volker Lendecke at 2018-11-01T16:34:31Z
torture: Fix the clang build

It's used uninitialized if an early torture_assert fails

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Thu Nov  1 17:34:31 CET 2018 on sn-devel-144

- - - - -
5528a4cc by Tim Beale at 2018-11-01T19:38:14Z
replmd: Add more debug for replicating links

During a join of a large DB, processing the linked attributes can take a
long time. The join hangs in 'Committing SAM database' for many minutes
with no indication of whether it's making progress or not.

This patch adds some extra debug to show how far through processing the
linked attributes we are, when there are many thousands of links.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
628b9f71 by Tim Beale at 2018-11-01T19:38:14Z
replmd: Group together link attribute processing by source object

Instead of processing each link attribute one at a time, we want to
group them together by source object. This will mean we only have to
look-up the source object once, and only perform one DB 'modify'
operation. With groups with 1000s of members, this will help improve
performance.

This patch takes the first step of group together the links by
source-object. A new 'la_group' struct is added to help track what links
belong to the same source object. The la_list essentially becomes a
'list of lists' now.

Note that only related links *in the same chunk* are only grouped together.
While it is trivial to groups together links that span different
replication chunks, this would be a fairly insignificant efficiency gain,
but seems to have a fairly detrimental memory overhead, once you get
into groups with 10,000+ members.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0ba02868 by Tim Beale at 2018-11-01T19:38:14Z
replmd: Split apart source/target checks for links

We've grouped the linked attributes by source-object. Next, we want to
avoid duplicated processing for the source object, i.e. we only need to
check the source object exists once, not once per link.

Before we can do this, we need to tease apart
replmd_extract_la_entry_details(), which is doing both source and target
object processing. Split out extracting the target DSDB-DN so that it's
done separately.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d54956fd by Tim Beale at 2018-11-01T19:38:14Z
replmd: Move talloc context one level up

Eventually we want to combine multiple link attributes, that apply to the
same source object, into a single DB 'modify' operation. This will mean
the memory context needs to hang around until we have performed the DB
operation (instead of allocating a temporary context for each link).

This patch moves the talloc context one level up, so a temp context gets
allocated for each link *group*, instead of for each link *attribute*.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
775054af by Tim Beale at 2018-11-01T22:48:21Z
replmd: Single DB operation per source object during link processing

Move the source object checks and DB modify operation up a level, so we
only do them once per source object rather than once per link.

This allows LMDB joins to succeed with ~15,000 members in a group.
Previously LMDB would fail with the error:

 Failed to apply linked attribute change '(-30792) - MDB_MAP_FULL:
  Environment mapsize limit reached at ../lib/ldb/ldb_mdb/ldb_mdb.c:203'

Rewriting the same object ~15000 times seemed to completely fill up
the LMDB 8Gb buffer. Presumably this was because LMDB is 'copy on
write', so it was storing ~15,000 copies of the same object. Strangely,
we don't see this problem writing the backlinks (which this patch won't
have helped with at all, because that's modifying the target object).

Note uSNChanged was only being added to the msg once, so the code has
been modified to replace the usnChanged each time (i.e. remove it and
re-add it).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Nov  1 23:48:21 CET 2018 on sn-devel-144

- - - - -
4731c338 by Gary Lockyer at 2018-11-01T22:49:24Z
windbindd: reword error message

Reword the asprintf() out of memory message to make it clear where the
issue is.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
d6777a66 by Gary Lockyer at 2018-11-01T22:49:24Z
source4 smbd: Make the service_details structure constant.

Make the service_details structure a static const.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
99aea425 by Gary Lockyer at 2018-11-01T22:49:24Z
source4 smdb: Add a post fork hook to the service API

Add a post fork hook to the service API this will be called:

 - standard process model
   immediately after the task_init.

- single process model
  immediately after the task_init

- prefork process model, inhibit_pre_fork = true
  immediately after the task_init

- prefork process model, inhibit_pre_fork = false
  after each service worker has forked. It is not run on the service
  master process.

The post fork hook is not called in the standard model if a new process
is forked on a new connection. It is instead called immediately after
the task_init.

The task_init hook has been changed to return an error code. This ensures
the post_fork code is only run if the task_init code completed successfully.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
7366de33 by Gary Lockyer at 2018-11-01T22:49:25Z
source4 smbd kdc: allow the kdc to run in prefork

Modify the kdc to allow it to run in the prefork process model. The
task_init function has been split up and code moved into the post_fork
function.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
325e928d by Gary Lockyer at 2018-11-01T22:49:25Z
kdc: Update debug calls

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
28826ec4 by Gary Lockyer at 2018-11-02T02:47:57Z
WHATSNEW: KDC prefork support

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Fri Nov  2 03:47:57 CET 2018 on sn-devel-144

- - - - -
f17a77af by Douglas Bagnall at 2018-11-02T02:48:52Z
python dbcheck: don't use mutable default args

In this code

def f(a, b=[]):
    b.append(a)
    return b

all single argument calls to f() will affect the same copy of b.

In the controls case, controls=None has the same effect as
controls=[].

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
1cf142c3 by Douglas Bagnall at 2018-11-02T02:48:52Z
selftesthelpers: fix py3 tests with extra_path

If a test was supplied with extra_path, a PYTHONPATH= env variable was
prepended to the args list, but the py3_compatible clause assumed the
first args element was /usr/bin/python.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
78f5b6e3 by Douglas Bagnall at 2018-11-02T02:48:52Z
s4/scripting/*: py3 compatible print

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <noel.power at suse.com>

- - - - -
537a26d6 by Noel Power at 2018-11-02T05:56:24Z
tests/py/rodc_rwdc: Fix py2/py3 .next compat issues

Python 3 does not have .next(), which we rely on, change the
code slightly so it works in py2/py3 (using builtin next function)

Pair-programmed-with: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Nov  2 06:56:24 CET 2018 on sn-devel-144

- - - - -
831ee63f by Volker Lendecke at 2018-11-02T16:03:26Z
lib: Add error checks in dom_sid_string_buf

Also, avoid casts by using PRIxxx macros

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8b9d3622 by Volker Lendecke at 2018-11-02T19:11:11Z
lib: Add dom_sid_str_buf

This is modeled after server_id_str_buf, which as an API to me is easier to
use: I can rely on the compiler to get the buffer size right.

It is designed to violate README.Coding's "Make use of helper variables", but
as this API is simple enough and the output should never be a surprise at all,
I think that's worth it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Nov  2 20:11:11 CET 2018 on sn-devel-144

- - - - -
94ad5ee6 by Ralph Boehme at 2018-11-02T20:21:13Z
libcli: add smbXcli_req_endtime

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e7eec24d by Ralph Boehme at 2018-11-02T20:21:13Z
libcli: fill endtime if smbXcli_req_create() timeout is non-zero

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a6de555c by Ralph Boehme at 2018-11-02T20:21:13Z
s4:libcli/smb2: reapply request endtime

tevent_req_finish() removed a possible request timeout, make sure to
reinstall it. This happened when an interim SMB2 response was received.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
35f9bc06 by Ralph Boehme at 2018-11-02T20:21:14Z
vfs_delay_inject: implement pread_send and pwrite_send

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e37ff8c5 by Ralph Boehme at 2018-11-02T20:21:14Z
s4:torture/smb2/read: add test for cancelling SMB aio

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
07eb805c by Ralph Boehme at 2018-11-02T20:21:14Z
s3:smbd: fix SMB2 aio cancelling

As we currently don't attempt to cancel the internal aio request, we
must ignore the SMB2 cancel request and continue to process the SMB2
request, cf MS-SM2 3.3.5.16:

  If the target request is not successfully canceled, processing of the
  target request MUST continue and no response is sent to the cancel
  request.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3fa45900 by Ralph Boehme at 2018-11-02T20:21:14Z
s3:smbd: remove now unused check if fsp is NULL

This was used internally to mark an aio request as cancelled. As the aio
cancellation functionality has been removed, we can now also remove this
check.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13667

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
07d05202 by Volker Lendecke at 2018-11-02T20:21:14Z
nbt_server: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b9c590a6 by Volker Lendecke at 2018-11-02T20:21:14Z
rpc_server4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
46026053 by Volker Lendecke at 2018-11-02T20:21:14Z
dsdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4c1d1076 by Volker Lendecke at 2018-11-02T20:21:14Z
auth: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
71549537 by Volker Lendecke at 2018-11-02T20:21:15Z
idmap: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dbc9a1a7 by Volker Lendecke at 2018-11-02T20:21:15Z
net: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d0a56a55 by Volker Lendecke at 2018-11-02T20:21:15Z
audit_tests: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
602d247e by Volker Lendecke at 2018-11-02T20:21:15Z
auth4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
bd1e43f7 by Volker Lendecke at 2018-11-02T20:21:15Z
winbindd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
528170ef by Volker Lendecke at 2018-11-02T20:21:15Z
winbindd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ce18cd2a by Volker Lendecke at 2018-11-02T20:21:15Z
smbd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
039aef70 by Volker Lendecke at 2018-11-02T20:21:15Z
libsmb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
be6f45ca by Volker Lendecke at 2018-11-02T20:21:15Z
lib: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7963e409 by Volker Lendecke at 2018-11-02T20:21:16Z
passdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
74c7e3d1 by Volker Lendecke at 2018-11-02T20:21:16Z
audit_logging: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ab542ed2 by Volker Lendecke at 2018-11-02T20:21:16Z
auth: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7d958162 by Volker Lendecke at 2018-11-02T20:21:16Z
lib: Use dom_sid_str_buf

This is the one place where we have to do another strpcy. Many of the
sid_to_fstring calls should be replacable by dom_sid_str_buf, so this
will get less.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c2b9b574 by Volker Lendecke at 2018-11-02T23:23:31Z
lib: Make dom_sid_string_buf static

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Nov  3 00:23:31 CET 2018 on sn-devel-144

- - - - -
dc9bbbe4 by Christof Schmitt at 2018-11-03T02:01:25Z
smbtorture: Add test for DELETE_ON_CLOSE on files with READ_ONLY attribute

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13673

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
162a5257 by Christof Schmitt at 2018-11-03T02:01:25Z
smbd: Fix DELETE_ON_CLOSE behaviour on files with READ_ONLY attribute

MS-FSA states that a CREATE with FILE_DELETE_ON_CLOSE on an existing
file with READ_ONLY attribute has to return STATUS_CANNOT_DELETE. This
was missing in smbd as the check used the DOS attributes from the CREATE
instead of the DOS attributes on the existing file.

We need to handle the new file and existing file cases separately.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13673

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a8e79dec by Christof Schmitt at 2018-11-03T02:01:25Z
selftest: Add share to test "delete readonly" option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13673

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7dd3585f by Christof Schmitt at 2018-11-03T04:55:45Z
selftest: Run smb2.delete-on-close-perms also with "delete readonly = yes"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13673

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Sat Nov  3 05:55:45 CET 2018 on sn-devel-144

- - - - -
e3e84b0f by Tim Beale at 2018-11-04T22:55:15Z
traffic_replay: Add helper class for group assignments

Wrap up the group assignment calculations in a helper class. We're going
to tweak the internals a bit in subsequent patches, but the rest of the
code doesn't really need to know about these changes.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
18740ec0 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Split out random group membership generation logic

This doesn't change functionality at all. It just moves the probability
calculations out into separate functions.

We want to tweak the logic/implementation behind this code, but the
rest of assign_groups() doesn't really care how the underlying
probabilities are worked out, so long as it gets a suitably random
user/group membership each time round the loop.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
898e6b43 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Improve assign_groups() performance with large domains

When assigning 10,000 users to 15 groups each (on average),
assign_groups() would take over 30 seconds. This did not include any DB
operations whatsoever. This patch improves things, so that it takes less
than a second in the same situation.

The problem was the code was looping ~23 million times where the
'random.random() < probability * 10000' condition was not met. The
problem is individual group/user probabilities get lower as the number
of groups/users increases. And so with large numbers of users, most of
the time the calculated probability was very small and didn't meet the
threshold.

This patch changes it so we can select a user/group in one go, avoiding
the need to loop multiple times.

Basically we distribute the users (or groups) between 0.0 and 1.0, so
that each user has their own 'slice', and this slice is proporational to
their weighted probability. random.random() generates a value between
0.0 and 1.0, so we can use this to pick a 'slice' (or rather, we use
this as an index into the list, using .bisect()). Users/groups with
larger probabilities end up with larger slices, so are more likely to
get picked.

The end result is roughly the same distribution as before, although the
first 10 or so user/groups seem to get picked more frequently, so the
weighted-probability calculations may need tweaking some more.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fdd75407 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Change user distribution to use Pareto Distribution

The current probability we were assigning to users roughly approximates
the Pareto Distribution (with shape=1.0). This means the code now uses a
documented algorithm (i.e. explanation on Wikipedia). It also allows us
to vary the distribution by changing the shape parameter.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5ad7fc73 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Prevent users having 1000+ memberOf links

When adding 10,000 users, one user would end up in over 1000 groups.
With 100,000 users, it would be more like 10,000 groups. While it makes
sense to have groups with large numbers of users, having a single user
in 1000s of groups is probably less realistic.

This patch changes the shape of the Pareto distribution that we use to
assign users to groups. The aim is to cap users at belonging to at most
~500 groups. Increasing the shape of the Pareto distribution pushes the
user assignments so they're closer to the average, and the tail (with
users in lots of groups) is not so large).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a29ee3a7 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Re-organize assignments to be group-based

We can speed up writing the group memberships by adding multiple users
to a group in a single DB modify operation.

To do this, we first need to reorganize the assignments so instead
of being a set of tuples, it's a dictionary where key=group and
value=list-of-users-in-group.

add_users_to_groups() now iterates through the users/groups slightly
differently, but mostly it's just indentation changes. We haven't
changed the number of DB operations yet - we'll do that in the next
patch.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
32e58227 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Write group memberships once per group

Each user-group membership was being written to the DB in a single
operation. With large numbers of users (e.g. 10,000 in average 15 groups
each), this becomes a lot of operations (e.g. 150,000). This patch
reworks the code so that we write the memberships for a group in
one operation. E.g. instead of 150,000 DB operations, we might make
1,500. This makes writing the group memberships several times
faster.

Note that rthere is a performance vs memory tradeoff. When we hit
10,000+ members in a group, memory-usage in the underlying DB modify
operation becomes very inefficient/costly. So we avoid potential memory
usage problems by writing no more than 1,000 users to a group at once.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
71c66419 by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Convert print() to logger.info()

Using logger is more helpful here because it includes timestamps, so we
can see how long things are taking. It's also more consistent with the
rest of the traffic_replay logging.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1906312c by Tim Beale at 2018-11-04T22:55:16Z
traffic_replay: Improve user generation debug

When creating 1000s of users you currently get a lot of debug, but at
the same time you have no idea how far through creating the users you
actually are.

Instead of logging every single user account that's created, log every
50th (as well as how far through the overall generation we are).

Logger already includes timestamps, so we can remove generating the
timestamp diff manually. User creation is the slowest operation - adding
groups/memberships is much faster, so we don't need to log as
frequently.

Note that there is a usability trade-off on how frequently we log
depending on whether the user is using the slower (but more common)
method of going via LDAP, vs the much faster (but more obscure) method
of writing directly to sam.ldb with ldb:nosync=true. In my tests, we end
up logging every ~30-ish secs with LDAP, and every ~3 seconds with
direct file writes.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
be51b512 by Tim Beale at 2018-11-04T22:55:17Z
traffic_replay: Generate machine accounts as well as users

Currently the tool only generates the machine accounts needed for
traffic generation. However, this isn't realistic if we're trying to use
the tool to generate users to simulate a large network.

This patch generates machine accoutns along with the user accounts.
Note we assume there will be more computer accounts than users in a real
network (e.g. work laptops, servers, etc), so generate slightly more
computer accounts.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
3338a3e2 by Tim Beale at 2018-11-05T02:43:24Z
traffic: Machine accounts were generated as critical objects

Due to the userAccountControl flags we were specifying, the machine
accounts were all created as critical objects. When trying to populate
1000s of machine accounts in a DB, this makes replication unnecessarily
slow (because it has to replicate them all twice).

This patch changes it so when we're just creating machine accounts for
the purpose of populating a semi-realistic DB, we jsut use the default
WORKSTATION_TRUST_ACCOUNT flag.

Note that for the accounts used for traffic-replay, we apparently need
the existing flags in order for the DC to accept certain requests.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Mon Nov  5 03:43:24 CET 2018 on sn-devel-144

- - - - -
27df4f00 by Martin Schwenke at 2018-11-05T05:52:33Z
ctdb-recovery: Ban a node that causes recovery failure

... instead of applying banning credits.

There have been a couple of cases where recovery repeatedly takes just
over 2 minutes to fail.  Therefore, banning credits expire between
failures and a continuously problematic node is never banned,
resulting in endless recoveries.  This is because it takes 2
applications of banning credits before a node is banned, which
generally involves 2 recovery failures.

The recovery helper makes up to 3 attempts to recover each database
during a single run.  If a node causes 3 failures then this is really
equivalent to 3 recovery failures in the model that existed before the
recovery helper added retries.  In that case the node would have been
banned after 2 failures.

So, instead of applying banning credits to the "most failing" node,
simply ban it directly from the recovery helper.

If multiple nodes are causing recovery failures then this can cause a
node to be banned more quickly than it might otherwise have been, even
pre-recovery-helper.  However, 90 seconds (i.e. 3 failures) is a long
time to be in recovery, so banning earlier seems like the best
approach.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13670

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Nov  5 06:52:33 CET 2018 on sn-devel-144

- - - - -
0934fc14 by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: PY2/PY3 compat changes for __init__.py

Fixes.

1) sorting of xml.etree.ElementTree.Element, in PY2 sort
   seems to sort lists of these. In PY3 this no longer works.
   Choosing tag as the sort key for py3 so at least in python3
   there is a consistent sort (probably won't match how it is
   sorted in PY2 but nothing seems to depend on that)
2) md5 requires bytes
3) tostring returns bytes in PY3, adjust code for that

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
388bddf4 by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: PY3 file -> open

'file' no longer exists in PY3 replace with 'open'

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d40ef736 by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: PY3 open file non-binary mode for write_binary

Although this is unintuitive it's because we are writing unicode
not bytes (both in PY2 & PY3). using the 'b' mode causes an error in
PY3.

In PY3 we can define the encoding, but not in PY2.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
cf79e6ae by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: PY2/PY3 compat porting for gp_init.py

Fixes
1) use compat versions of ConfigParser and StringIO
2) fix sort list of XML Elements
3) open file needs to be opened in binary mode as write_pretty_xml
   routing uses BytesIO() object.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
df578e15 by Noel Power at 2018-11-05T19:05:23Z
python/samba/gp_parse: Use csv.reader for parsing cvs files

The previous version here was using UnicodeReader which was
wrapping the UTF8Recoder class and passing that to csv.reader.
It looks like the intention was to read a bytestream in a
certain encoding and then reencode it to a different encoding.
And then UnicodeReader creates unicode from the newly encoded stream.
This is unnecssary, we know the encoding of the bytesstream and
codec.getreader will happily consume the bytstream and give back
unicode. The unicode can be fed directly into csv.writer.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
54e2bb70 by Noel Power at 2018-11-05T19:05:23Z
python/samab/gp_parse: remove unused code

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
19a459ba by Noel Power at 2018-11-05T19:05:24Z
python/samba/netcmd: misc PY2/PY3 compat changes for gpo.py

Fixes:
1) various ldb.bytes that should be displayed as strings in PY3
2) sorting of lists of xml Element in PY3
3) various 'open' need to be opened in binary mode (to accept binary
   data)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
16596842 by Noel Power at 2018-11-05T19:05:24Z
python/samba/gp_parse: Fix mulitple encode step with write_section

In python2 as far as I can see GptTmplInfParser.write_binary more
or less works by accident.

write_binary creates a writer for the 'utf8' codec, such a writer
should consume unicode and emit utf8 encoded bytes. This writer
is passed to each of the sections managed by GptTmplInfParser as
follows

    def write_binary(self, filename):
        with codecs.open(filename, 'wb+',
                         self.encoding) as f:
            for s in self.sections:
                self.sections[s].write_section(s, f)

And each section type itself is encoding its result to 'utf-16-le'
e.g.
    class UnicodeParam(AbstractParam):
         def write_section(self, header, fp):
            fp.write(u'[Unicode]\r\nUnicode=yes\r\n'.encode(self.encoding)

But this makes little sense, it seems like sections are encoded to one
encoding but the total file is supposed to be encoded as ut8??? Also
having an encoding per ParamType doesn't seem correct.

Bizarely in PY2 this works and it actually encodes the whole file as utf-16le
In PY3 you can't do this as the writer wants to deal with strings not bytes
(after the extra encode phase in 'write_section'.

So, changes here are to remove the unnecessary encoding in each 'write_section'
method, additionally in GptTmplInfParser.write_binary the
codecs.open call now uses the correct codec (e.g. 'utf-16-le') to write

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
6476ef58 by Noel Power at 2018-11-05T19:05:24Z
python/samba/tests/samba_tool: PY2/PY3 compat port for test

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
fc047c2c by Noel Power at 2018-11-05T19:05:24Z
python/samba/gp_parse: PY2/PY3 Decode only when necessary

In python2 we decode str types in load_xml, in python3 these are
str class(s) which we cannot decode.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e355a6bc by Noel Power at 2018-11-05T22:04:48Z
s4/selftest: enable samba.tests.samba_tool.gpo for PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Mon Nov  5 23:04:48 CET 2018 on sn-devel-144

- - - - -
02d41fea by Tim Beale at 2018-11-06T02:39:11Z
join: Fix join large-DB timeout against Windows

The LDAP connection can also timeout when trying to join a Windows DC
with a very large database. However, in this case Windows gives a
slightly different error message (NT_STATUS_CONNECTION_RESET instead of
NT_STATUS_CONNECTION_DISCONNECTED).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13612

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c21a8ed5 by Tim Beale at 2018-11-06T02:39:11Z
libnet: Reset debug counters after replicating critical objects

Reset the debug counters once we have finished replicating a given
partition. This helps if we replicate the same partition immediately
afterward with different options.

This helps the DC join debug look less weird. Because it replicates the
critical objects first, and then the base partition, previously it
always ended up overcounting, e.g.

Partition[DC=addom,DC=samba,DC=example,DC=com] objects[314/218]
  linked_values[48/24]

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
63bfdb3c by Tim Beale at 2018-11-06T02:39:11Z
drs_utils: Fix some long lines

Tweak the code slightly to avoid some 80+ character lines.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2229f462 by Tim Beale at 2018-11-06T06:15:33Z
drs_utils: Avoid invalid dereference of v8 requests

req.more_flags only exists for v10 requests, so we throw an exception if
we try to dereference that field on a v8 (or v5) request. Unfortunately,
we were checking that we support v10 *after* we had tried to access the
more_flags. This patch fixes up the order of the checks.

This may be a problem trying to replicate with an older Windows DC
(pre-2008R2), and was reported on the samba mailing-list at one point:
https://lists.samba.org/archive/samba/2018-June/216541.html

Unfortunately this patch doesn't help the overall situation at all (the
join will fail because we can't resolve the link target and we can't use
GET_TGT). But it now gives you a more meaningful error, i.e.

  ERROR(runtime): uncaught exception - (8639, "Failed to process 'chunk'
    of DRS replicated objects: DOS code 0x000021bf"
instead of:
  ERROR(<type 'exceptions.AttributeError'>): uncaught exception -
    'drsuapi.DsGetNCChangesRequest8' object has no attribute 'more_flags'

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Tue Nov  6 07:15:33 CET 2018 on sn-devel-144

- - - - -
32c2ec8f by Martin Schwenke at 2018-11-06T06:16:13Z
ctdb-common: Allow path_socket() to use $CTDB_SOCKET

Use of CTDB_SOCKET is being generally removed.  However, this override
is being added to allow test code outside of ctdb/ to be able to
specify the socket, if desired.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d4a1f897 by Martin Schwenke at 2018-11-06T06:16:13Z
ctdb-tests: Use ctdb-path-like values for local daemons socket and PID file

However, don't use ctdb-path itself because some tests use nested
instances of onnode.  The outermost instance would set CTDB_SOCKET and
any inner instance would pick up that value, regardless of CTDB_BASE.

This is a temporary measure to avoid breaking testing while use of the
path functions is added to ctdbd and the ctdb tool.  When this is
complete these variables can be removed altogether because the code
will just depend on CTDB_TEST_MODE and CTDB_BASE.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
38566780 by Martin Schwenke at 2018-11-06T06:16:13Z
ctdb-tests: Use ctdb-path for fake_ctdbd directory setup

This needs to be done before any of the code changes are made,
including updating the ctdb tool.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
cc3aedd3 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tools: Use path_socket() in ctdb tool

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
cd021596 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Use path_socket() in test client tools

Just leak the memory allocated by path_socket().  This is only used in
short-lived test programs, so it isn't worth the hassle of plumbing a
talloc context through several layers to get here.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
5f478b7c by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-daemon: Use path functions for socket and PID file

Drop the use of ctdb_set_sockname() because it complicates the memory
allocation and this is the only place it is used.  Just assign to the
relevant pointer.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d75fa2c3 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-daemon: Drop unused function ctdb_set_socketname()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
82e589e3 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Drop setting of CTDB_SOCKET and CTDB_PIDFILE

The local daemons ssh stub doesn't need to do this because the ctdbd
and the ctdb tool now only need CTDB_TEST_MODE and CTDB_BASE for local
daemon tests.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
3b1e5977 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Drop incorrect comment, unused function

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d246b1da by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Use path_socket() in dummy client

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4f4a835c by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: Remove export of CTDB_SOCKET

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
08469408 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tests: README updates

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
1bbc4fad by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-tools: Detect unknown node number

If there aren't enough addresses in the list then the shift will
silently fail and the printed address will be the unshifted value of
$1, which is incorrect/unexpected.  So, sanity check the node number.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
c84254d2 by Martin Schwenke at 2018-11-06T06:16:14Z
ctdb-daemon: Avoid unnecessarily spamming the logs when in test mode

Logging the logging location to syslog can be useful on production
systems when the configuration goes unexpectedly missing.  However, in
test mode this just adds noise to the logs on the test system.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
9c41481f by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-daemon: Don't set log_to_stdout for become_daemon()

ctdbd logs to stderr in interactive mode, not stdout.  This way stdout
is always closed.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
4e6bd424 by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-daemon: Improve documentation for -i option

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e952f031 by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-event: Never fork to become daemon in eventd

This stops ctdbd from being able to shut down eventd, since the PID it
records will be invalid.  There's no need for eventd to fork.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
44019b55 by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-event: Only run talloc report if CTDB_INTERACTIVE is set

This is only really wanted for interactive testing when logging to
stderr.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
f420e8d5 by Jeremy Allison at 2018-11-06T06:16:15Z
lib: talloc: Mark talloc_autofree_context() as deprecated.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
698a9ef3 by Jeremy Allison at 2018-11-06T06:16:15Z
lib: ldb: Remove use of talloc_autofree_context().

Just use NULL in test case. talloc_autofree_context() is deprecated.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
eda1296d by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-tests: Add function wait_until_node_has_no_ips()

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
5fdac517 by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-tests: Use wait_until_node_has_no_ips() in some tests

This strengthens those tests to ensure that released IPs aren't
replaced with others.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
ba86eacb by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-tests: Rationalise ctdb stop/continue/disable/enable simple tests

The "continue" and "enable" tests are just extensions of the "stop"
and "disable" tests, so drop the latter 2.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
43c26e1e by Martin Schwenke at 2018-11-06T06:16:15Z
ctdb-tests: Rationalise tunable simple tests

These 3 tests duplicate various checks and can easily be handled as a
single test.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
9ebcebe5 by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop useless "ctdb version" test

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
52056805 by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop use of confusing testfailures variable

Exit on first test failure instead of setting a variable.  The bizarre
logic in ctdb_test_exit() makes this worth dropping.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
6d9c89bf by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop setting of unused variable scriptname

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
e733e4cb by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Ignore SIGPIPE during simple test cleanup

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
0e9ead8f by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Start daemons in ctdb_test_init(), stop them in ctdb_test_exit()

This makes tests self-contained.  They can also now be individually
looped, if necessary.

Most tests (all but 1 complex, more than 50% of simple) restart the
daemons anyway, so this simplification is worth it.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
92337234 by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop cleanup_handler()

Running testsuite-specific code here isn't a good option.

Daemons are now shut down in ctdb_test_exit(), even when testing is
interrupted.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
aa2ee4be by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Improve signal handling trap

Interrupting a test run currently moves on to the next test.  It
should exit.

Follow the practice of exiting with 128 + signal number.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
412fb7b7 by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Move enabling of event scripts to setup_ctdb()

This is for the real cluster case.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
bc8df719 by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop tests that only start and stop daemons

Integration test cases now start and shut down the cluster.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
79db138d by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop dependency on variable ctdb_test_restart_scheduled

The remainder of the scheduled restart logic is about to be removed,
so produce debugging information any time the cluster is not healthy.

While here, reindent and drop the else since there is already an early
return before it.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
d762e52e by Martin Schwenke at 2018-11-06T06:16:16Z
ctdb-tests: Drop ctdb_restart_when_done()

This no longer does anything.  Integration test cases now start and
shut down the cluster.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
9a2910c6 by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-tests: Drop passing of test arguments to ctdb_test_init()

Arguments are currently ignored anyway.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2cb82ef4 by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-tests: Change all cluster setup to use ctdb_test_init()

ctdb_test_init() now passes any arguments to setup_ctdb().

Update tests that have custom local daemon configuration to call
ctdb_test_init() directly.  Remove the redundant, initial call to
ctdb_test_init() to avoid starting the cluster an extra time.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
bde73c7a by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-tests: Add new variable CTDB_SCRIPTS_TESTS_BINDIR

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
7ac5e7ae by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-tests: Use $CTDB_SCRIPTS_TESTS_BINDIR

Don't calculate this locally as _test_bin_dir.  Just calculate
top_dir, source script_install_paths.sh and use
$CTDB_SCRIPT_TESTS_BINDIR.

Don't bother sanity checking if TEST_BIN_DIR is set.  It will go away
soon.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
1d86fd53 by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-tests: Use $CTDB_SCRIPTS_TOOLS_BIN_DIR

Don't calculate this locally as _tools_dir.  Add it to PATH
unconditionally - this may result in duplicate entries in PATH but the
resulting code is simpler.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
aa0c7ccf by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-tests: Move setting of ctdb_dir and top_dir

These are only used in script_install_paths.sh.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
9679c9e9 by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-build: Don't set unused variable TEST_BIN_DIR

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
25efb924 by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-tests: Allow use of setup_ctdb_base() outside of test cases

Always create an empty event script directory.  If $TEST_SUBDIR is
unset then don't use it to look for etc-ctdb/.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
46fd4f14 by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-tests: Add local_daemons.sh

This provides a separate script for handling local daemons.  It can be
used for testing outside of the CTDB simple test suite.  It is
installed as ctdb_local_daemons.

The logic is copied from ctdb/tests/simple/scripts/local_daemons.bash.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
19de5f46 by Martin Schwenke at 2018-11-06T06:16:17Z
ctdb-tests: Use local_daemons.sh in local_daemons.bash

The etc-ctdb/ subdirectory containing the event script moves into the
top-level tests/ directory because the subdirectory is really now
owned by local_daemons.sh instead of simple/.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
804bdf97 by Martin Schwenke at 2018-11-06T06:16:18Z
ctdb-tests: Add local_daemons.sh onnode and socket commands

These aren't used by simple tests but they will be useful for
integrating ctdbd local daemons into other test suites and for
debugging.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
01f6fbba by Martin Schwenke at 2018-11-06T06:16:18Z
ctdb-daemon: Switch interactive variable to a bool

popt uses an int in place of a bool, so declare an extra int and make
the conversion explicit.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
6e16e95f by Martin Schwenke at 2018-11-06T09:30:14Z
ctdb-daemon: Do not fork when CTDB_TEST_MODE is set

Explicitly background ctdbd instead.

This has the advantage of leaving stdin open.  ctdbd can then be
enhanced to exit when stdin closes, allowing better cleanup in a test
environment.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Tue Nov  6 10:30:14 CET 2018 on sn-devel-144

- - - - -
46a87f2c by Volker Lendecke at 2018-11-06T17:57:25Z
tdb: Add tdb_traverse_chain

This is a lightweight readonly traverse of a single chain, see the
comment in the header file.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3daf9c85 by Volker Lendecke at 2018-11-06T17:57:25Z
tdb: Add test for tdb_traverse_chain

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e814fb6b by Volker Lendecke at 2018-11-06T17:57:26Z
tdb: Version 1.3.17 for tdb_traverse_chain

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
72ec893d by Volker Lendecke at 2018-11-06T17:57:26Z
tdb: Allow !CLEAR_IF_FIRST & MUTEX_LOCKING

This is a prerequisite to allow gencache to run on a non-transactioned
database with mutexes.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0d7f67f0 by Volker Lendecke at 2018-11-06T17:57:26Z
gencache: Convert to a binary timestamp

Two reasons:

The ascii conversion shows up on profiles.

In a further commit we will get checksums for gencache entries to
protect at hidden corruption due to a crash on the non-transactioned
gencache.tdb. Next to the timestamp this is a second field that is
gencache metadata, and I don't want to deal with a second ascii number
when at least some of the gencache values are binary already.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a1e13b4a by Volker Lendecke at 2018-11-06T17:57:26Z
gencache: Add crc check

This covers key, timestamp and data. This will detect silent
corruption of gencache data after a system crash

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1386200b by Volker Lendecke at 2018-11-06T17:57:26Z
gencache: Remove transaction-based tdb

At more than one large site I've seen significant problems due to
gencache_stabilize. gencache_stabilize was mainly introduced to
survive machine crashes with the cache still being in place. Given
that most installations crash rarely and this is still a cache, this
safety is overkill and causes real problems.

With the recent changes to tdb, we should be safe enough to run on
completely corrupted databases and properly detect errors. A further
commit will introduce code that wipes the gencache.tdb if such a
corruption is detected.

There is one kind of corruption that we don't properly handle:
Orphaned space in the database. I don't have a good idea yet how to
handle this in a graceful and efficient way during normal operations,
but maybe this idea pops up at some point.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c13eb552 by Volker Lendecke at 2018-11-06T17:57:26Z
gencache: Wipe corrupt databases

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
608df97d by Volker Lendecke at 2018-11-06T17:57:26Z
gencache: Prune expired entries

This solves the problem that gencache never shrinks right
now. Whenever we write an entry, we now walk that entry's chain and
delete expired entries. This should be a good balance between
performance and cleanup actions: Reading is still unaffected, and
those who write pay a small penalty while keeping gencache size under
control.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8d787f73 by Volker Lendecke at 2018-11-06T20:52:32Z
tdb: Align integer types

tdb->max_dead_records is "int", as is the corresponding parameter to
tdb_set_max_dead(). Not that a signed variable makes any sense, but
this is old code and tdb_set_max_dead() is a public API which we
should not change for this.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Nov  6 21:52:32 CET 2018 on sn-devel-144

- - - - -
5870c642 by Tim Beale at 2018-11-07T00:29:23Z
selftest: Add README documenting the customdc testenv

Also documented the other backup/restore testenvs as well.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
3ca13997 by Tim Beale at 2018-11-07T03:39:05Z
selftest: Add some more testenv descriptions

This still doesn't cover all the testenvs comprehensively, but it
pretty much exhausts my knowledge of what the various testenvs do.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Wed Nov  7 04:39:05 CET 2018 on sn-devel-144

- - - - -
a9682f94 by Douglas Bagnall at 2018-11-07T16:55:08Z
python/upgradehelpers: use mkstemp, not mktemp

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e3320b6d by Douglas Bagnall at 2018-11-07T16:55:08Z
samba-tool gpo: do not use predictable temp directory

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
94610330 by Douglas Bagnall at 2018-11-07T16:55:08Z
samba-tool gpo: add helper method for tmpdir construction

A few of the gpo commands use an identical temporary directory structure
that can be constructed using shared code.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fd089c37 by Douglas Bagnall at 2018-11-07T16:55:08Z
samba-tool gpo: convert pseudo-method into method

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
85b6d889 by Tim Beale at 2018-11-07T16:55:08Z
traffic_replay: Move machine account creation

I was assuming that generate_users_and_groups() only gets called in the
--generate-users-only case. However, it also gets called in the default
traffic replay case.

This patch reworks the code so that the number of machine accounts to
create gets passed in, and the 'create 25% more computers than users'
assumption only applies to the --generate-users-only case.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
51917fc0 by Tim Beale at 2018-11-07T16:55:08Z
traffic_replay: Move 'traffic account' flag up a level

We create machine accounts for 2 different purposes:
1). For traffic generation, i.e. testing realistic network packets.
2). For generating a realistic large DB.

Unfortunately, we want to use different userAccountControl flags for
the 2 different cases. Commit 3338a3e257fa9f28 changed the flags used
for case #2, but this breaks case #1.

The problem is generate_users_and_groups() is called in both cases,
so we want the 'traffic account' flag passed into that function.
This ensures that the machine accounts get created with the appropriate
userAccountControl flags for the particular case you want to test.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c7fe4814 by Tim Beale at 2018-11-07T16:55:09Z
traffic_replay: Make sure naming assumptions are in a single place

The traffic_replay group/user/machine account names follow a standard
format. This adds a function to generate the machine-name. It also makes
sure the existing user_name() function gets called in all applicable
places.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1d7fb66f by Tim Beale at 2018-11-07T16:55:09Z
traffic_replay: Make packet generation work on a pre-populated DB again

Generate separate machine accounts for populating a large DB vs
replaying network traffic.

We want to use different userAccountControl flags in each of the above
cases (i.e. commit 3338a3e257fa9f28). However, this means that once you
use the --generate-users-only option, you can't replay network packets
against the machine accounts.

We can avoid this problem by creating separate machine accounts for each
of 2 different cases, e.g. STGM-0-x machines for traffic-replay, and
PC-0-x machines for padding out the database.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a0b5f4b7 by Tim Beale at 2018-11-07T16:55:09Z
traffic_replay: Rework machine accounts to remove redundant code

generate_users_and_groups() now generates the machine acounts as well as
the user accounts, so it seems there's no need to also have
generate_traffic_accounts(), which does the same job.

Instead, we can just pass through the number of machine acounts to
generate_users_and_groups() and delete the other function.

Also updated generate_users_and_groups() so that machine_accounts is
no longer optional (we want to create machine accounts in all cases).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
de1f4975 by Gary Lockyer at 2018-11-07T16:55:09Z
autobuild: make ad_dc use prefork process model

Convert the ad_dc environment from single process model to the prefork
process model.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
700b4ce9 by Gary Lockyer at 2018-11-07T16:55:09Z
source4 smbd prefork: Increase default worklers to 4

Increase the default number of worker processes started by the pre-fork
process model from 1 to 4.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b543563d by Gary Lockyer at 2018-11-07T16:55:09Z
WHATSNEW: 'prefork children' default

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b003134e by Gary Lockyer at 2018-11-07T16:55:09Z
source4 smbd: pass instance number to post_fork

Pass the instance number to the post_fork hook. This is required to
allow the rpc_server to support the prefork process model.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
58c4490e by Gary Lockyer at 2018-11-07T16:55:09Z
source4 smdb rpc_server: Support prefork process model.

Allow the rpc_server to run in the prefork process model. Due to the use
of shared handles and resources all of the rpc end points are serviced
in the first worker process. Those end points that can be run in
multiple processes (currently only Netlogon and management) are serviced in
the first and any subsequent workers.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e650c40e by Gary Lockyer at 2018-11-07T20:17:59Z
WHATSNEW: NETLOGON prefork support

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Nov  7 21:17:59 CET 2018 on sn-devel-144

- - - - -
9f4b4002 by Andreas Schneider at 2018-11-08T07:07:10Z
nsswitch:tests: Pass the envname to the script

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
71ef09c1 by Andreas Schneider at 2018-11-08T07:07:10Z
lib:util: Fix DEBUGCLASS pointer initializiation

This fixes a segfault in pyglue:

==10142== Process terminating with default action of signal 11 (SIGSEGV)
==10142==  Bad permissions for mapped region at address 0x6F00A20
==10142==    at 0x6F1074B: py_set_debug_level (pyglue.c:165)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13679

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2d512b27 by Andreas Schneider at 2018-11-08T10:03:11Z
debug: Use debuglevel_(get|set) function

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Thu Nov  8 11:03:11 CET 2018 on sn-devel-144

- - - - -
593fdd21 by Volker Lendecke at 2018-11-08T23:47:22Z
smbd: Add "hide new files" option

See the manpage for the description

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f21bc3ad by Volker Lendecke at 2018-11-09T02:49:55Z
selftest: Test hide new files timeout

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Nov  9 03:49:55 CET 2018 on sn-devel-144

- - - - -
f8f2c562 by Philipp Gesang at 2018-11-09T16:52:30Z
s4-kdc: restore MIT KDC backend

Fix fallout from the KDC prefork patchset (99aea42520fc..).

GCC warns when Samba is being built with --with-system-mitkrb5.
Fix this by adapting the signature of mitkdc_task_init() to match
task_init which has been extended to return a status code.

Status codes try to mimick those of kdc-heimdal.c:kdc_task_init()
as closely as possible.

Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
96d47c21 by Tim Beale at 2018-11-09T16:52:30Z
selftest: Add README note: always use --configfile in testenv creation

We always need to specify "env->{CONFIGURATION}" when running a samba-tool
command to setup a testenv. Add a note to the README as this wasn't at
all clear.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
55fa7bc0 by Tim Beale at 2018-11-09T16:52:30Z
selftest: Fix backup testenv creation on certain host machines

When we created the backup-file for the restoredc/renamedc/labdc
testenvs we weren't explicitly a --configfile on the samba-tool command.
This meant the command tried to use the smb.conf form the default
install location, i.e. /usr/local/samba/etc/smb.conf. On the gitlab CI
runner, there's no samba installed, so it ends up using the default
settings, which is fine. However, if the host machine had an invalid
smb.conf installed there, creating the testenv would fail with an error
like:

ERROR(runtime): uncaught exception - Unable to load default file
File "bin/python/samba/netcmd/__init__.py", line 184, in _run
return self.run(*args, **kwargs)
File "bin/python/samba/netcmd/domain_backup.py", line 222, in run
lp = sambaopts.get_loadparm()
  File "bin/python/samba/getopt.py", line 94, in get_loadparm
    self._lp.load_default()

We can avoid this by always explictly specifying the backupfromdc's
smb.conf when creating the backup file.

Likewise, labdc/customdc also need the config specified when the admin
password is reset.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
b161b3a8 by Tim Beale at 2018-11-09T20:07:47Z
mdb_util: Better error message if lmdb-utils not installed

mdb_copy() was dutifully checking the PATH for the mdb_copy executable,
then, if it didn't find it, blindly proceeding anyway and trying to run
a non-existent executable. This resulted in a cryptic error:

  ERROR(<type 'exceptions.OSError'>): uncaught exception - [Errno 2] No
    such file or directory

Add in an extra check that we actually find the executable and raise a
better human-readable exception if we don't.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Nov  9 21:07:47 CET 2018 on sn-devel-144

- - - - -
9b303504 by Ralph Wuerthner at 2018-11-09T21:42:27Z
nsswitch: Fix CID 1441072 Error handling issues (CHECKED_RETURN)

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5d53870d by Ralph Wuerthner at 2018-11-09T21:42:27Z
nsswitch: Fix CID 1441070 Error handling issues (CHECKED_RETURN)

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
221133b0 by Ralph Boehme at 2018-11-09T21:42:27Z
s4:torture/vfs/fruit: torture writing AFP_AfpInfo stream

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13677

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4901d71c by Ralph Boehme at 2018-11-09T21:42:27Z
vfs_fruit: move a comment to the right place

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13677

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a7c87784 by Ralph Boehme at 2018-11-09T21:42:27Z
vfs_fruit: validation of writes on AFP_AfpInfo stream

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13677

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e8743444 by Volker Lendecke at 2018-11-09T21:42:28Z
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
21bfc774 by Volker Lendecke at 2018-11-09T21:42:28Z
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
58e77b54 by Volker Lendecke at 2018-11-09T21:42:28Z
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1ff8598d by Volker Lendecke at 2018-11-09T21:42:28Z
libcli: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
efcd3ae6 by Volker Lendecke at 2018-11-09T21:42:28Z
passdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
94cfb4de by Volker Lendecke at 2018-11-09T21:42:29Z
passdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9060b26c by Volker Lendecke at 2018-11-09T21:42:29Z
passdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
beba440c by Volker Lendecke at 2018-11-09T21:42:29Z
passdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f564776e by Volker Lendecke at 2018-11-09T21:42:29Z
idmap: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1e484da7 by Volker Lendecke at 2018-11-09T21:42:29Z
idmap: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
87ae595b by Volker Lendecke at 2018-11-09T21:42:29Z
idmap: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5cad0fc6 by Volker Lendecke at 2018-11-09T21:42:29Z
libgpo: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a979bb28 by Volker Lendecke at 2018-11-10T01:19:29Z
lib: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Nov 10 02:19:29 CET 2018 on sn-devel-144

- - - - -
899d00e6 by Justin Stephenson at 2018-11-10T11:48:02Z
s4-auth-tests: Fix test_kerberos with MIT krb

When a keytab of type MEMORY is used, the MIT kerberos krb5_kt_add_entry()
library function adds a keytab entry to the beginning of the keytab table,
instead of the end.

This adds a MIT kerberos conditional to reverse iterate through
the keytable entries to address this.

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Alexander Bokovoy <ab at samba.org>
Autobuild-Date(master): Sat Nov 10 12:48:02 CET 2018 on sn-devel-144

- - - - -
c8e27b6f by Andreas Schneider at 2018-11-12T08:50:25Z
lib:replace: Add getprogname()

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ba9ad126 by Andreas Schneider at 2018-11-12T08:50:25Z
wbclient: Send the client process name talking to winbind

This is for better debugging messages.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
305f3bc4 by Andreas Schneider at 2018-11-12T08:50:25Z
s3:winbindd: Also log the process name in winbindd

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d26caa27 by Andreas Schneider at 2018-11-12T08:50:25Z
s3:winbind: Log client process name

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ec87ca8b by Andreas Schneider at 2018-11-12T08:50:25Z
s3:winbind: Log client process name in winbind_misc

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f49b62b4 by Andreas Schneider at 2018-11-12T08:50:25Z
s3:winbind: Log client process name for PAM auth

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1834513e by Andreas Schneider at 2018-11-12T08:50:25Z
wbclient: Add wbcSetClientProcessName()

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6f257f95 by Andreas Schneider at 2018-11-12T08:50:25Z
krb5_plugin: Set the wbclient process name

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2a918a64 by Andreas Schneider at 2018-11-12T08:50:26Z
nss_winbind: Set the client process name

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ffc8b4ac by Andreas Schneider at 2018-11-12T08:50:26Z
nss_wins: Set client process name

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c10d70c4 by Andreas Schneider at 2018-11-12T12:44:09Z
pam_winbind: Set the request type as client name

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Mon Nov 12 13:44:09 CET 2018 on sn-devel-144

- - - - -
751c0cc7 by Volker Lendecke at 2018-11-12T19:20:52Z
winbindd: Fix crash when taking profiles

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13629
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Björn Baumbach <bbaumbach at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Mon Nov 12 20:20:52 CET 2018 on sn-devel-144

- - - - -
71671549 by Volker Lendecke at 2018-11-13T03:10:54Z
smbd: Use wire_perms_to_unix in unix_perms_from_wire

Same code, less lines

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Nov 13 04:10:54 CET 2018 on sn-devel-144

- - - - -
7a9d003d by Andreas Schneider at 2018-11-13T06:37:25Z
lib:replace: Do not leak the file pointer in rep_getprogname()

And return NULL on error.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
63a9fe1d by Andreas Schneider at 2018-11-13T06:37:25Z
nsswitch: Handle possible NULL return value of getprogname()

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
9f576810 by Martin Schwenke at 2018-11-13T10:11:11Z
replace: Add check for variable program_invocation_short_name

It appears that wafsamba's configure() defines _GNU_SOURCE
unconditionally, so checking _GNU_SOURCE isn't enough to know if this
variable is available.

For example, it isn't available on AIX with the xlc compiler:

  [ 6/10] Compiling lib/replace/replace.c
  ...
  "../../lib/replace/replace.c", line 991.16: 1506-045 (S) Undeclared identifier program_invocation_short_name.

Instead, add a configure check for program_invocation_short_name and
use it.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Tue Nov 13 11:11:11 CET 2018 on sn-devel-144

- - - - -
d0a8899e by Ralph Boehme at 2018-11-13T10:13:02Z
s3:selftest: split "raw.session" and "smb2.session"

The next commit is going to add a testsuite to "smb2.session".

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b86c94f0 by Ralph Boehme at 2018-11-13T10:13:03Z
s3:selftest: also run smb2.session torture testsuite against ad_member

The next commit adds a subtest to the smb2.session testsuite that
requires Kerberos (ad_dc would work), but where neither SMB2 server or
client must require signing (ad_dc, being an AD DC, requires signing).

The ad_member environment supports Kerberos with the SMB2 server not
mandating signing, that'll do.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
d407201d by Ralph Boehme at 2018-11-13T10:13:03Z
libcli/smb: add smb2cli_session_require_signed_response()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
67cfb016 by Ralph Boehme at 2018-11-13T10:13:03Z
libcli/smb: maintain require_signed_response in smbXcli_req_state

Not used for now, that comes next.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
7abf3900 by Ralph Boehme at 2018-11-13T10:13:03Z
libcli/smb: defer singing check a little bit

This allows adding an additional condition to the if check where the
condition state may be modified in the "if (opcode ==
SMB2_OP_SESSSETUP)" case directly above.

No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
53fe1484 by Ralph Boehme at 2018-11-13T10:13:03Z
libcli/smb: use require_signed_response in smb2cli_conn_dispatch_incoming()

This can be used by the upper layers to force checking a response is
signed. It will be used to implement verification of session setup
reauth responses in a torture test. That comes next.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
368e1860 by Ralph Boehme at 2018-11-13T10:13:03Z
s4:torture/smb2/session: invalidate credential cache

Invalidate credential cache before connecting to the server, otherwise
we will reuse the credentials from the credential cache populated by the
preceeding tests.

Also invalidate it at the end, otherwise subsequent tests might run into
problems if the credentials expire while authenticating.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
ffc424ee by Ralph Boehme at 2018-11-13T10:13:03Z
s4:torture/smb2/session: require a signed session setup reauth response

All existing tests using this function require signing, so currently
this passes. A subsequent commit adds a test where neither client nor
server require signing and that's where this trap will explode.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
5fdea409 by Ralph Boehme at 2018-11-13T10:13:03Z
s4:torture/smb2/session: add force_signing to test_session_expire1i

Existing callers pass true, so no change in behaviour. The next commit
adds an additional test that passes force_signing=false.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
181f18c4 by Ralph Boehme at 2018-11-13T10:13:04Z
s4:torture/smb2/session: session reauth response must be signed

This test checks that a session setup reauth is signed even when neither
client nor server require signing.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
bb93e691 by Ralph Boehme at 2018-11-13T13:22:46Z
s3:smb2_sesssetup: check session_info security level before it gets talloc_move'd

We talloc_move() session_info to session->global->auth_session_info
which sets session_info to NULL.

This means security_session_user_level(NULL, NULL) will always return
SECURITY_ANONYMOUS so we never sign the session setup response.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13661

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Tue Nov 13 14:22:46 CET 2018 on sn-devel-144

- - - - -
a5d1bb5c by Ralph Boehme at 2018-11-13T13:32:15Z
s4:torture/smb2/session: test smbXcli_session_set_disconnect_expired() works

This adds a simple test that verifies that after having set
smbXcli_session_set_disconnect_expired() a session gets disconnected
when it expires.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=9175

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
5a8583ed by Ralph Boehme at 2018-11-13T16:28:45Z
libcli/smb: don't overwrite status code

The original commit c5cd22b5bbce724dcd68fe94320382b3f772cabf from bug
9175 never worked, as the preceeding signing check overwrote the status
variable.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=9175

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Tue Nov 13 17:28:45 CET 2018 on sn-devel-144

- - - - -
3634e20c by Jeremy Allison at 2018-11-13T19:54:56Z
s3: lib: nmbname: Ensure we limit the NetBIOS name correctly. CID: 1433607

Firstly, make the exit condition from the loop explicit (we must
never write into byte n, where n >= sizeof(name->name).

Secondly ensure exiting from the loop that n==MAX_NETBIOSNAME_LEN,
as this is the sign of a correct NetBIOS name encoding (RFC1002)
in order to properly read the NetBIOS name type (which is always
encoded in byte 16 == name->name[15]).

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>

Autobuild-User(master): David Disseldorp <ddiss at samba.org>
Autobuild-Date(master): Tue Nov 13 20:54:56 CET 2018 on sn-devel-144

- - - - -
56ac8944 by Andreas Schneider at 2018-11-14T04:07:15Z
s3:rpcclient: Initialize domain_name

This could be passed uninitialized to dcerpc_netr_DsRGetDCName()

Found by cppcheck.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13680

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9c37ed26 by Andreas Schneider at 2018-11-14T04:07:15Z
librpc:ndr: Initialize inblob

Found by cppcheck.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13680

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
159f7537 by Andreas Schneider at 2018-11-14T04:07:15Z
libgpo: Make sure status is intialized

Found by cppcheck.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13680

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
050d1e7d by Andreas Schneider at 2018-11-14T04:07:15Z
lib:util Always initialize start and space

Found by cppcheck.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13680

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
008b9652 by Andreas Schneider at 2018-11-14T04:07:15Z
ctdb: Fix an out of bound array access

Found by cppcheck.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13680

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5b4c6648 by Noel Power at 2018-11-14T04:07:15Z
python/samba/provision: PY3 port samba4.dlz_bind9.version

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0ef6e3da by Noel Power at 2018-11-14T04:07:16Z
testprogs/blackbox: PY3 bulk change for python scripts use correct python

Change all instance where python scripts are called so that the
correct python version as specified by $PYTHON is used

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
10810936 by Noel Power at 2018-11-14T04:07:16Z
python/samba/tests: Port samba.tests.blackbox.samba_dnsupdate for PY2/PY3

Make sure either the output of tests and/or the item we are searching match
in type. Output of cmd in python3 is bytes, depending on the was the test is
written it may be easier just to convert all output or just a single string
that is used in the test

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b16043f5 by Noel Power at 2018-11-14T04:07:16Z
python/samba/tests: PY3 port samba.tests.blackbox.netads_json

convert output from check_output to text
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a2536768 by Noel Power at 2018-11-14T04:07:16Z
s3/torture: PY3 port samba.ntlm_auth

Use bytes in test rather than str

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fdb3aa48 by Noel Power at 2018-11-14T04:07:16Z
python/samba/netcmd: PY3 port samba.tests.samba_tool.edit

Need to write bytes to file

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
32a490d6 by Noel Power at 2018-11-14T04:07:16Z
python/samba/netcmd: PY3 port samba4.blackbox.ldapcmp_restore

Convert attribute from bytes to str
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
17158129 by Noel Power at 2018-11-14T04:07:16Z
s4/dsdb/tests/python: PY3 port samba4.ldap_schema.python

+ Misc attributes needed to be converted to strings from bytes to
ensure various asserts work as expected.
+ Fix ndr_unpack call which needs bytes not str

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e6ea052c by Noel Power at 2018-11-14T04:07:17Z
s4/dsdb/tests/python: PY3 port samba4.ldap.secdesc

User str/bytes as needed for various asserts

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fe859a44 by Noel Power at 2018-11-14T04:07:17Z
s4/dsdb/tests: PY3 port of samba4.ldap.acl.python

convert various attribute results to str so assert function
as expected for tests.
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f176461c by Noel Power at 2018-11-14T04:07:17Z
python/samba: PY3 fix can't compare string with int

Testing a string against an int value is illegal and
is not necessary in this case, this patch removes the
problematic test.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8bf82d9f by Noel Power at 2018-11-14T04:07:17Z
python/samba/tests: PY3 Fix str/bytes issue for json.loads

Python 3.4 seems to need a string

    parsed = json.loads (out_jsobj)
  File "/usr/lib/python3.4/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

however Python 3.5 seems to be happy to consume bytes (or string)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
48cc8a4e by Tim Beale at 2018-11-14T04:07:17Z
s4/scripting/bin: PY3 convert servicePrincipalName attr to string

res[0]["servicePrincipalName"] is an instance of ldb.bytes in PY3
If we wish to get the string value we need to call the custom
str function which attempts to decode the bytes to 'utf8'

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed by: Noel Power <npower at samba.org>

- - - - -
a5c8ab27 by Noel Power at 2018-11-14T04:07:17Z
python/samba/tests: Ensure samba_dnsupdate called with correct python version

We need to examine the contents of PYTHON env variable which should defined the
python version to be used when running tests.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3495f3fc by Noel Power at 2018-11-14T04:07:17Z
s4/scripting/bin: gensec_client.update needs bytes

PY3 test was failing as param passed to update was str rather than bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2ca3b3eb by Noel Power at 2018-11-14T04:07:17Z
script: Add new (temporary) pure python3 ad-dc-2 test

Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
321002e3 by Noel Power at 2018-11-14T04:07:18Z
CI: Add new CI job for new purepy3-ad-dc-2 autobuild task

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
483d5ea5 by Noel Power at 2018-11-14T07:20:55Z
CI: Autobuild: Remove build_samba_ad_dc_2_py3 CI job

We now run a purepython3 ad-dc-2 test job, later when the whole
build is running under python3 we will resurrect build_samba_ad_dc_2_py3
but as (build_samba_ad_dc_py2) for python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Nov 14 08:20:55 CET 2018 on sn-devel-144

- - - - -
0595129b by Andreas Schneider at 2018-11-14T07:45:19Z
libldb: Fix const char in pyldb

../lib/ldb/pyldb.c: In function ‘PyDict_AsMessage’:
../lib/ldb/pyldb.c:90:22: error: initialization discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
 #define PyStr_AsUTF8 PyUnicode_AsUTF8
                      ^~~~~~~~~~~~~~~~
../lib/ldb/pyldb.c:1359:19: note: in expansion of macro ‘PyStr_AsUTF8’
   char *key_str = PyStr_AsUTF8(key);
                   ^~~~~~~~~~~~
../lib/ldb/pyldb.c: In function ‘py_ldb_msg_getitem_helper’:
../lib/ldb/pyldb.c:3336:7: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
  name = PyStr_AsUTF8(py_name);
       ^
../lib/ldb/pyldb.c: In function ‘py_ldb_msg_setitem’:
../lib/ldb/pyldb.c:3502:12: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
  attr_name = PyStr_AsUTF8(name);
            ^

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Uri Simchoni <uri at samba.org>

- - - - -
50fdf67b by Andreas Schneider at 2018-11-14T07:45:19Z
s4:librpc: Use discard_const_p for ndr/py_misc

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Uri Simchoni <uri at samba.org>

- - - - -
2a6e7f3a by Andreas Schneider at 2018-11-14T07:45:19Z
s4:dsdb: Use const char in py_dsdb_garbage_collect_tombstones()

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Uri Simchoni <uri at samba.org>

- - - - -
4a272b83 by Andreas Schneider at 2018-11-14T10:46:06Z
s3:passdb: Use discard_const_p() in py_passdb

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Uri Simchoni <uri at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Wed Nov 14 11:46:06 CET 2018 on sn-devel-144

- - - - -
a4a85aca by Jeremy Allison at 2018-11-14T20:35:45Z
s3: VFS: Remove logically dead code. Coverity CID: 1419117

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Böhme <slow at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Nov 14 21:35:45 CET 2018 on sn-devel-144

- - - - -
ce6b5919 by Andreas Schneider at 2018-11-15T00:47:23Z
s4:torture: Use 65520 for maxopenfiles

The socket_wrapper limit is 65535 open sockets.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e3b6402b by Andreas Schneider at 2018-11-15T03:50:31Z
third_party: Update socket_wrapper to version 1.2.1

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Nov 15 04:50:31 CET 2018 on sn-devel-144

- - - - -
c1dd6382 by Martin Schwenke at 2018-11-15T07:15:32Z
ctdb-tests: Make the debug hung script test cope with unreadable stacks

Ideally this would just involve using "test -r".  However, operating
system security features may mean that kernel stacks are not readable
even though they appear to be.

Instead, try reading that stack of a process on the test node.  If
that succeeds then so should reading the stack of the "stuck" sleep
process in the test.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13684

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Tim Beale <timbeale at catalyst.net.nz>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Thu Nov 15 08:15:32 CET 2018 on sn-devel-144

- - - - -
9b79d5f2 by Jeremy Allison at 2018-11-15T15:30:17Z
s3: VFS: fake_acls: Fix Coverity CID 1435850 Uninitialized pointer read.

map_acl_perms_to_permset() can return an error, check it.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
0872f140 by Volker Lendecke at 2018-11-15T18:45:24Z
torture: Fix the 32-bit build

Unfortunately there's no off_t printf specifier as there's one for
size_t. So we have to use intmax_t.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Thu Nov 15 19:45:24 CET 2018 on sn-devel-144

- - - - -
da1d7af1 by Volker Lendecke at 2018-11-16T17:35:22Z
lib: Fix CID 1441264 Error handling issues (CHECKED_RETURN)

This is not worth keeping a CID around :-)

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
579ce3eb by Volker Lendecke at 2018-11-16T17:35:22Z
dsdb: Slightly simplify samdb_check_password

Avoid an "else" where we have the early return

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Björn Baumbach <bbaumbach at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
be46a427 by Christof Schmitt at 2018-11-16T20:52:13Z
smbtorture: Close unused pipe fds in kernel_oplocks8

This fixes a hang of the testcase when hitting an error in the child (e.g.
localdir does not exist)

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Nov 16 21:52:13 CET 2018 on sn-devel-144

- - - - -
cc9f1d46 by Tim Beale at 2018-11-20T00:33:33Z
WHATSNEW: Added samba-tool 'backup offline' and 'groups stats'

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
65e7476a by Tim Beale at 2018-11-20T00:33:33Z
tests: Add assertion that replUpToDateVector is present after backup

We noticed that offline backups were missing a replUpToDateVector for
the original DC, if the backup was taken on a singleton DC. This patch
adds an assertion to the existing test-cases to highlight the problem.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6e5ebbfb by Tim Beale at 2018-11-20T00:33:33Z
netcmd: Add backupType marker to backed-up DB

We are starting to hit restore cases that are only applicable to a
particular type of backup. We already had a marker to differentiate
renames, but differentiating offline backups would also be useful.

Note that this raises a slight compatibility issue for backups created
on v4.9, as the marker won't exist. However, it's only offline backups
we will use this marker for (at the moment), and this option doesn't
exist on v4.9, so there's no problem.

Removing the markers has been refactored out into a separate function to
handle the optional presence of the new marker.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5ca2726a by Tim Beale at 2018-11-20T00:33:33Z
netcmd: Small backup refactor to avoid compatiblity problems

It will be easy to forget that the backupType marker doesn't exist on
v4.9. However, this seems like a dumb reason not to support v4.9
backup-files. Add a wrapper function to avoid potential problems
cropping up in future.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
008449d9 by Tim Beale at 2018-11-20T00:33:33Z
netcmd: Flush replUpToDateVector when restoring offline backup

The replUpToDateVector could be incorrect after an offline backup was
restored. This means replication propagation dampening doesn't work
properly. In the worst case, a singleton DC would have no
replUpToDateVector at all, and so *all* objects created on that DC get
replicated every time a new DRS connection is established between 2 DCs.
This becomes a real problem if you used that singleton DC to create 100K
objects...

This patch flushes the replUpToDateVector when an offline backup gets
restored. We need to do this before we add in the new DC and remove the
old DCs.

Note that this is only a problem for offline backups. The online/rename
backups are received over DRS, and as part of the replication they
receive the latest replUpToDateVector from the DC being backed up.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
eea1d4b7 by Gary Lockyer at 2018-11-20T00:33:33Z
dsdb group_audit tests: fix use of strncmp

Replace the uses of:
   strncmp(expected, value, strlen(expected))
With:
    strcmp(expected, value)

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1dc06aaa by Stefan Metzmacher at 2018-11-20T00:33:33Z
s4:heimdal_build: make use of libreplace getprogname() replacement

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5bb48cb6 by Stefan Metzmacher at 2018-11-20T00:33:34Z
wafsamba: fix CHECK_MAKEFLAGS() with waf 2.0.8

Changing Options.options.jobs in the build() hook
is too late in waf 2.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1cb78211 by Stefan Metzmacher at 2018-11-20T00:33:34Z
wafsamba: add a fix for broken python threading if just one job is forced

This fixes random failures during (at least) configure on AIX.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c0104a4a by Stefan Metzmacher at 2018-11-20T00:33:34Z
wafsamba: remove hardcoded '..' and '/default/' from SAMBA_PIDL()

This makes it possible to remove some move waf 1.8 compat code.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8e1135f9 by Stefan Metzmacher at 2018-11-20T00:33:34Z
wafsamba: remove the need of BuildContext.bdir

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
076afab4 by Stefan Metzmacher at 2018-11-20T00:33:34Z
wafsamba: remove unused Build.BuildContext.pre_build overload

This is not needed and also fixed the interaction between
vim and ':make'

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
65f7f7ff by Stefan Metzmacher at 2018-11-20T00:33:34Z
wafsamba: simplify SAMBA_PIDL_TABLES() rule

The builddir is not bin/default/ instead of just bin/,
so we don't need to strip 'default/' anymore.

And the '--output ${TGT}' part is not really implemented.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
aafc15c3 by Stefan Metzmacher at 2018-11-20T00:33:35Z
librpc/tables.pl: remove unused $opt_output option

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0025e8e9 by Andreas Schneider at 2018-11-20T03:47:24Z
waf: Load the C compiler correctly

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Nov 20 04:47:24 CET 2018 on sn-devel-144

- - - - -
af1f24ac by Tim Beale at 2018-11-20T03:48:46Z
replmd: Split up replmd_verify_linked_attribute() into src/target checks

Refactor replmd_verify_linked_attribute() so we split out the link
attribute source/target checks. This patch should not alter
functionality.

The source object check has been moved out to where
replmd_verify_linked_attribute() was called.

replmd_verify_linked_attribute() has been renamed, as it's now only
checking the link target.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ffe77076 by Tim Beale at 2018-11-20T03:48:46Z
replmd: Skip redundant source object link checks

We receive the links grouped together by source object. We can save
ourselves some work by not looking up the source object for every single
link (if it's still the same object we're dealing with).

We've already made this change to replmd_process_linked_attribute().
This patch makes the same change to replmd_store_linked_attributes().
(We verify that we know about each link source/target as we receive each
replication chunk. replmd_process_linked_attribute() kicks in later as
the transaction completes).

Note some care is needed to hold onto the tmp_ctx/src_msg across
multiple passes of the for loop.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c49c0c30 by Tim Beale at 2018-11-20T03:48:46Z
replmd: Minor change to replmd_verify_link_target() args

We were passing in the entire src_msg, but all we really need is the
source object's DN (and even then, it's only used in error messages).

Change it so we only pass in what the function actually needs. This
makes it a bit easier to see what src_msg is actually used for.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
062ac12a by Tim Beale at 2018-11-20T03:48:46Z
replmd: Split some code out into create_la_entry() helper function

replmd_store_linked_attributes() has gotten in szie and complexity. This
refactors some code out into a separate function to make things a bit
more manageable.

This patch should not alter functionality.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8b47443b by Tim Beale at 2018-11-20T07:40:16Z
replmd: Cache recycle-bin state to avoid DB lookup

By caching the recycle-bin state we can save ~6% of the join time.

Checking whether the recycle-bin is enabled involves an underlying DSDB
search. We do this ~4 times for each link we replicate (twice for the
link source and target). By caching the recycle-bin's state over the
duration of the replication, we can save 1000s of unnecessary DB
searches.

With 5K users this makes the join time ~5 secs faster.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Tue Nov 20 08:40:16 CET 2018 on sn-devel-144

- - - - -
b7a0d3b1 by Garming Sam at 2018-11-20T21:14:16Z
dirsync: Allow arbitrary length cookies

The length of the cookie is proportional to the number of DCs ever in
the domain (as it stores the uptodateness vector which has stale
invocationID).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13686

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ac90c9fa by Garming Sam at 2018-11-20T21:14:16Z
sync_passwords: Remove dirsync cookie logging for continuous operation

Under normal operation, users shouldn't see giant cookies in their logs.
We still log the initial cookie retrieved from the cache database, which
should still be helpful for identifying corrupt cookies.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13686

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ad8bb6fc by Garming Sam at 2018-11-20T21:14:16Z
ldb_controls: Add some talloc error checking for controls

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13686

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9a1277c1 by Gary Lockyer at 2018-11-20T21:14:16Z
tests samr: Extra tests for samr_QueryDisplayInfo

Add extra tests to test the content returned by samr_QueryDisplayInfo,
which is not tested for the ADDC.  Also adds tests for the result
caching added in the following commit.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d9f7b6d6 by Gary Lockyer at 2018-11-20T21:14:17Z
tests samr: remove PEP8 warnings

Remove PEP8 warnings from the samr tests.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
10efdac3 by Gary Lockyer at 2018-11-20T21:14:17Z
source4 samr: cache samr_QueryDisplayInfo results

Add a cache of GUID's that matched the last samr_QueryDisplayInfo made on a
domain handle.  The cache is cleared if the requested start index is
zero, or if the level does not match that in the cache.

The cache is maintained in the guid_caches array of the dcesrv_handle.

Note: that currently this cache exists for the lifetime of the RPC
      handle.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fc1a16ff by Andrew Bartlett at 2018-11-20T21:14:17Z
s4-samr: Use dom_sid_split_rid() to get the RID in dcesrv_samr_QueryDisplayInfo

Signed-off-by: Andrew Bartlett <abartlet at samba.org>

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
7dd7800a by Gary Lockyer at 2018-11-20T21:14:17Z
test samr: Extra tests for samr_EnumDomainGroups

Add extra tests to test the content returned by samr_EnumDomainGroups,
and tests for the result caching added in the following commit.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
564813b5 by Gary Lockyer at 2018-11-20T21:14:17Z
source4 samr: cache samr_EnumDomainGroups results

Add a cache of GUID's that matched the last samr_EnunDomainGroups made on a
domain handle.  The cache is cleared if resume_handle is zero, and when the
final results are returned to the caller.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b6241852 by Gary Lockyer at 2018-11-20T21:14:17Z
s4-samr: Use dom_sid_split_rid() to get the RID in dcesrv_samr_EnumDomainGroups

Signed-off-by: Andrew Bartlett <abartlet at samba.org>

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
8d8303b0 by Andrew Bartlett at 2018-11-20T21:14:17Z
s4-samr: Use GUID_buf_string() in dcesrv_samr_EnumDomainGroups()

This avoids memory allocation

Signed-off-by: Andrew Bartlett <abartlet at samba.org>

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
fa3ea1cf by Gary Lockyer at 2018-11-20T21:14:17Z
tests samr: Extra tests for samr_EnumDomainUserss

Add extra tests to test the content returned by samr_EnumDomainUsers,
and tests for the result caching added in the following commit.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
64007aa3 by Gary Lockyer at 2018-11-20T21:14:17Z
source4 samr: cache samr_EnumDomainUsers results

Add a cache of GUID's that matched the last samr_EnunDomainUsers made on a
domain handle.  The cache is cleared if resume_handle is zero, and when the
final results are returned to the caller.

The existing code repeated the database query for each chunk requested.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
53c20e10 by Andrew Bartlett at 2018-11-20T21:14:18Z
s4-samr: Use dom_sid_split_rid() to get the RID in dcesrv_samr_EnumDomainUsers

Signed-off-by: Andrew Bartlett <abartlet at samba.org>

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
1719f8b9 by Andrew Bartlett at 2018-11-20T21:14:18Z
s4-samr: Use GUID_buf_string() in dcesrv_samr_EnumDomainUsers()

This avoids memory allocation.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>

Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
ad57cac7 by Gary Lockyer at 2018-11-21T00:50:11Z
source4 samr: Tidy DBG_WARNING calls

Move the calls to GUID_buf_string and dom_sid_str_buf into the
coresponding DBG_WARNING call, instead of using an intermediate variable.
While this violates the coding guidelines, doing this makes the code less
cluttred and means the functions are only called if the debug message is
printed.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Nov 21 01:50:11 CET 2018 on sn-devel-144

- - - - -
cb3520fb by Tim Beale at 2018-11-21T00:51:11Z
replmd: replmd_process_link_attribute() returns type of change made

In order to share work across related link attribute updates, we need
replmd_process_link_attribute() to let the caller know what actually
changed.

This patch adds an extra return type that'll be used in the next patch.
What we're interested in is: the update was ignored (i.e. it's old news),
a new link attribute was added (because this affects the overall
msg/element memory), and an existing link attribute was modified (due to
how links are actually stored, this includes deleting the link, as in
reality it simply involves setting the existing link to 'inactive').

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c371fef5 by Tim Beale at 2018-11-21T00:51:11Z
replmd: Only modify the object if it actually changed

Commit 775054afbe1512 reworked replmd_process_link_attribute() so that
we batch together DB operations for the same source object. However, it
was possible that the object had not actually changed at all, e.g.
- link was already processed by critical-objects-only during join, or
- we were doing a full-sync and processing info that was already
  up-to-date in our DB.

In these cases we modified the object anyway, even though nothing had
changed. This patch fixes it up, so we check that the object has
actually changed before modifying the DB.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f53954d0 by Tim Beale at 2018-11-21T00:51:11Z
replmd: Move where we update the usnChanged/whenChanged

Move this closer to where the source object actually gets modified.

The main reason to do this is that adding fields can cause the
msg->elements to be reallocated, which will invalidate all the
old_el and pdn_list pointers which are derived from the msg.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
19a36b36 by Tim Beale at 2018-11-21T00:51:11Z
replmd: Remove some redundant code

At first glance, this code seemed completely unnecessary. However, it
was added (by commit f6bc4c08b19f5615) for a valid reason: adding the
whenChanged/uSNChanged attributes to the message can cause msg->elements
to be reallocated, which means the old_el pointer (which points to
msg->elements memory) can be out of date.

whenChanged/uSNChanged now get added to the msg last, just before the DB
modify operation. So old_el can no longer become out of date within
replmd_process_link_attribute(), so re-fetching it is now redundant.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
90f5e49a by Tim Beale at 2018-11-21T00:51:11Z
replmd: Pass old_el into replmd_process_linked_attribute()

We should only need to lookup the msg attribute once per source object.
The old_el->values may change due to link-processing, but old_el itself
should not.

This is not aimed at improving performance, but we need to change how
old_el is used before we can change pdn_list (which is more costly
processing-wise).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
698cf271 by Tim Beale at 2018-11-21T00:51:11Z
replmd: Minimize get_parsed_dns_trusted() calls during replication

When a group has 10,000+ links, get_parsed_dns_trusted() can be costly
(simply the talloc calls alone are expensive). Instead of re-generating
the pdn_list for every single link attribute, we can change to only
re-generate it when we really need to.

When we add a new link, it reallocates old_el->values, and so we need to
recreate the pdn_list because all the memory pointers will have changed.
However, in the other cases, where we're simply updating the existing
link value (or ignoring the update, if it's already applied), we can
continue using the same pdn_list (rather than re-parsing it again).

This would generally only save time with a full-sync - it won't really
help with the join case (because every link processed results in a
realloc).

On a DB with 5000 users, this makes a full-sync about ~13% faster.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
05147d25 by Tim Beale at 2018-11-21T00:51:11Z
replmd: Avoid redundant dsdb_get_deleted_objects_dn() checks

Quite a bit of time was spent in dsdb_get_deleted_objects_dn()
processing during either a join (~9%) or a full-sync (~28%).

The problem is we're *always* doing the dsdb_get_deleted_objects_dn()
call for each object, regardless of whether it's actually deleted or
not. i.e. we were doing an expensive query and a lot of the time just
ignoring the query result.

If it's not a deleted object we're dealing with, we can just return
early and skip the unnecessary processing.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a370f217 by Tim Beale at 2018-11-21T04:31:10Z
replmd: Make replmd_process_linked_attribute() mem dependencies clearer

This patch should not alter functionality - it is just making memory
assumptions used in replmd_process_linked_attribute() clearer.

When adding/removing msg->elements we have to take care, as this will
invalidate things like the parsed-DN array or old ldb_message_element
pointers. This has always been the case (i.e. f6bc4c08b19f5615a49),
however, now we need to take even more care, as the msg being modified
is re-used and split across 2 different functions.

Add more code comments to highlight this. We can also free
pdn_list/old_el to prevent them being incorrectly used after realloc.
It seems appropriate to also add a sanity-check that the tmp_ctx alloc
succeeds (which all the other memory hangs off).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Wed Nov 21 05:31:10 CET 2018 on sn-devel-144

- - - - -
5f47c04c by Joe Guo at 2018-11-21T06:46:18Z
netcmd/ldapcmp: fix typo for Bundle

Bundel -> Bundle

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
683342b7 by Joe Guo at 2018-11-21T06:46:18Z
netcmd/ldapcmp: rename __eq__ to diff

This method actually changed both objects and print info.
__eq__ is not a proper name and is not designed for this case.
Rename to diff.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
263f0207 by Joe Guo at 2018-11-21T06:46:18Z
netcmd/ldapcmp: add choices arg to --scope option

So we don't need to validate ourselves.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e71d0d71 by Joe Guo at 2018-11-21T06:46:19Z
netcmd/ldapcmp: add choices arg to --view option

So we don't need to validate ourselves.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
86882bd1 by Joe Guo at 2018-11-21T06:46:19Z
netcmd/ldapcmp: avoid list comprehension in for loop

The list comprehension will repeat for each item.
For large database, this make the command freeze.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f821f84f by Joe Guo at 2018-11-21T06:46:19Z
netcmd/ldapcmp: use set instead of list to find missing DNs

This simplify the logic and improve performance a lot.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
04713870 by Joe Guo at 2018-11-21T06:46:19Z
netcmd/ldapcmp: rm unused global var summary

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bbd082e7 by Joe Guo at 2018-11-21T06:46:19Z
netcmd/ldapcmp: avoid modifying data while looping on dict

Just define another dict for return value, seems no need to modify
original dict.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2851cd5b by Joe Guo at 2018-11-21T06:46:19Z
netcmd/ldapcmp: pass --skip-missing-dn to LDAPBase

This option has default value False, and was actually not passed down from cli
to LDAPBase. However, LDAPBase.__init__ has default value True for it.

After the change, a few tests using ldapcmp are affected.
Add --skip-missing-dn explicitly to keep the behavior consistent,
otherwise test will fail.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
10855509 by Joe Guo at 2018-11-21T06:46:20Z
netcmd/ldapcmp: fix wrong way for string copy

Two mistakes here:

- res[:-1] will copy but lost the last char
- string is immutable in python, there is no need to copy it explicitly

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7d9282bf by Joe Guo at 2018-11-21T06:46:20Z
netcmd/ldapcmp: use set instead of list to compare attrs

This will simplify the logic and improve performance.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
26f0992c by Joe Guo at 2018-11-21T06:46:20Z
netcmd/ldapcmp: promote re object to global

Then we can reuse the re obj.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7ebbd373 by Joe Guo at 2018-11-21T06:46:20Z
netcmd/ldapcmp: make code pythonic

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c4bd785a by Douglas Bagnall at 2018-11-21T06:46:20Z
samba-tool ldapcmp: print DNs on predictable order

Rather than unstable hash order. Ideally we'd do them in proper DN order.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
24b1aa9d by Joe Guo at 2018-11-21T09:46:20Z
logger: use color automatically for a tty

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Nov 21 10:46:20 CET 2018 on sn-devel-144

- - - - -
bb843be3 by Volker Lendecke at 2018-11-21T15:53:40Z
libsmb: Make cli_trans a normal subreq function

A lot of "state" variables, but still I don't think the additional complexity
is justified by saving a few lines. Make this a completely regular tevent_req
"req/subreq" function.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9c9ccf23 by Volker Lendecke at 2018-11-21T15:53:41Z
libsmb: Make cli_trans_send cancellable

Required to cancel SMB1 notify

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
499d97b3 by Volker Lendecke at 2018-11-21T15:53:41Z
libcli: Make smb2cli_notify_send cancellable

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d4a8bed8 by Volker Lendecke at 2018-11-21T15:53:42Z
libsmb: Introduce cli_smb2_notify_send/recv

This only existed as a sync wrapper where the underlying function could do
send/recv already.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4c984f3f by Volker Lendecke at 2018-11-21T15:53:42Z
libsmb: Make cli_notify_send/recv smb2-aware

smb2-awareness only existed in the sync wrapper

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
88d82b44 by Volker Lendecke at 2018-11-21T19:50:13Z
libsmb: Make cli_notify_send cancellable

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Wed Nov 21 20:50:13 CET 2018 on sn-devel-144

- - - - -
02514427 by Swen Schillig at 2018-11-22T07:22:18Z
libads: set proper ads_keytab_flush() return code on error

The return code was left on success when the calls to
ads_get_machine_kvno() or ads_clear_service_principal_names()
failed and the processing had to be aborted.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
a3dd28c6 by Swen Schillig at 2018-11-22T07:22:18Z
smbspool: Free kerberos context on error

If processing cannot continue and the function returns
the kerberos context must be free'd.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
1295e2b7 by Volker Lendecke at 2018-11-22T07:22:18Z
README.Coding: Approve DBG statements using dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
7f902798 by Swen Schillig at 2018-11-22T07:22:18Z
s3: Free principal if smb_krb5_principal_get_realm() fails

If smb_krb5_principal_get_realm() fails, procesing is aborted and
resources have to be free'd. In this context free'ing the principal
was missing.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
1a6451de by Swen Schillig at 2018-11-22T07:22:18Z
librpc: Free krb5 context on error

If the call to krb5_cc_resolve() fails and processing is aborted,
the krb5 conext must be free'd before return.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
c73c345f by Swen Schillig at 2018-11-22T07:22:19Z
utils: Free krb5 context for net_lookup_kdc

In case of an error while processing and at the end of processing
the krb5 conext must be free'd.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
64fbc4c1 by Swen Schillig at 2018-11-22T10:18:41Z
utils: Free host realm after processing

In case of error and at the end of processing the list
of realms have to be free'd again.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Thu Nov 22 11:18:41 CET 2018 on sn-devel-144

- - - - -
3e58e83b by Andreas Schneider at 2018-11-22T21:13:26Z
s4:bind_dlz: Only copy the dn once

../source4/dns_server/dlz_bind9.c:1132:4: error: 'dn' may be used uninitialized in this function [-Werror=maybe-uninitialized]
    state->log(ISC_LOG_INFO, "failed to find dnsRecord for %s",
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        ldb_dn_get_linearized(dn));
        ~~~~~~~~~~~~~~~~~~~~~~~~~~

Found by Undefined Sanitizer.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
dbeb806d by Andreas Schneider at 2018-11-22T21:13:26Z
s4:torture: Initialize pointers in libnetapi user test

Found by Undefined Sanitizer.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
dbdd57e3 by Andreas Schneider at 2018-11-22T21:13:26Z
s3:tests: Initialize pointers with NULL in netdisplay test

Found by Undefined Sanitizer

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
2a418f73 by Andreas Schneider at 2018-11-22T21:13:27Z
lib:util: Always include unistd.h for setgroups

This is needed to pass configure checks

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
1642dc2f by Andreas Schneider at 2018-11-22T21:13:27Z
s3:lib: Do not redefine bool and use stdbool.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
ee649757 by Andreas Schneider at 2018-11-22T21:13:27Z
lib:crypto: Fix undefined behavior in md4

runtime error: left shift of 145 by 24 places cannot be represented in type 'int'

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
502f56c1 by Andreas Schneider at 2018-11-22T21:13:27Z
s3:lib: Fix undefined behavior in messages_dgm

source3/lib/messages_dgm.c:1290:7: runtime error: variable length array
bound evaluates to non-positive value 0

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
02913d08 by Andreas Schneider at 2018-11-22T21:13:27Z
lib:util: Fix undefined behavior in asn1 parser

lib/util/asn1.c:969 runtime error: left shift of negative value -1

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
470a9b89 by Andreas Schneider at 2018-11-22T21:13:27Z
lib:util: Fix undefined behavior in idtree

lib/util/idtree.c:84 runtime error: left shift of 1 by 31 places cannot
be represented in type 'int'

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
93ab0cef by Andreas Schneider at 2018-11-22T21:13:27Z
lib:util: Fix undefined behavior in bitmap.c

lib/util/bitmap.c:77: runtime error: left shift of 1 by 31 places cannot
be represented in type 'int'

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
aab5034a by Andreas Schneider at 2018-11-22T21:13:27Z
lib:compression: Fix undefined behavior in lzxpress

lib/compression/lzxpress.c:228 runtime error: store to misaligned
address 0x5631d53ca9fe for type 'uint32_t', which requires 4 byte
alignment

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
6f48bc84 by Andreas Schneider at 2018-11-23T00:23:09Z
librpc:ndr: Fix undefined behavior in ndr.c

librpc/ndr/ndr.c:1430 runtime error: left shift of 1 by 31 places cannot
be represented in type 'int'

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Fri Nov 23 01:23:09 CET 2018 on sn-devel-144

- - - - -
b8ff4a08 by Tim Beale at 2018-11-23T04:00:22Z
tests: Add better error for DRS tests

We've got a flappy test hitting this assertion failure, but we can't
tell why it's failing intermittently (probably because we're bumping the
RID-Set, but there's no way to confirm this).

Add some extra debug info if the test assertion fails.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1a45e5b1 by Tim Beale at 2018-11-23T04:00:22Z
tests: Fix flappiness in DRS tests due to RID Set changing

The test_link_utdv_hwm test case in getnc_exop has started getting
slightly flappy (8 failures in the last 2 weeks). The problem is the
test case creates a new computer, which can occasionally result in a new
RID pool being allocated.

The problem can be reproduced by running the test case repeatedly (it
usually fails after ~250 times).

This patch updates the _check_ctr6() assertion to filter out the 'CN=RID
Set' object, if it happens to be present.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b9c5417b by Tim Beale at 2018-11-23T04:00:22Z
replmd: Avoid passing msg to replmd_process_linked_attribute()

We can prevent anyone from inadvertently adding/removing msg->elements[]
in replmd_process_linked_attribute() by just not passing msg into the
function. Currently we only actually need the source DN and a memory
context for reallocating old_el->values.

The warning comment has been moved to a more appropriate place.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9d5af87e by Tim Beale at 2018-11-23T04:00:22Z
replmd: Move logic into new replmd_should_apply_isDeleted() function

It's easier to follow the logic involved here when it's split out into a
separate function.

This patch should not alter the existing logic/functionality.

Note the 'else' case is somewhat redundant, but it avoids excessive
whitespace changes to the function. It'll be tidied up in the next
patch.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6b136a81 by Tim Beale at 2018-11-23T07:10:41Z
replmd: remove unnecessary indent

The previous refactor now means we return early if we don't need to
re-apply isDeleted to the object. The 'else' is redundant and we can
remove it to avoid unnecessary indent.

This patch is basically just a whitespace change. It should not alter
functionality.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Nov 23 08:10:41 CET 2018 on sn-devel-144

- - - - -
5fa134dc by Gary Lockyer at 2018-11-23T07:25:19Z
source4 smbd test: prefork process restart

Add tests for the restarting of failed/terminated process, by the
pre-fork process model.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1adb5b21 by Gary Lockyer at 2018-11-23T07:25:19Z
source4 smbd prefork: Pass restart information

Pass information about the pre-fork master and worker processes that
will allow them to be restarted.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d4641c8f by Gary Lockyer at 2018-11-23T07:25:19Z
source4 smbd prefork: Restart failed processes

Restart any pre-forked master or worker process that terminated with SIGABRT,
SIGBUS, SIGFPE, SIGILL or SIGSYS

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3315a28e by Gary Lockyer at 2018-11-23T07:25:19Z
source4 smbd process: pass the fatal flag to terminate

Pass the fatal flag supplied to task_server_terminate to the process
task_terminate method.  It will be used by the task_terminate methods to
set an appropriate exit code.  The process_prefork model will use a non
zero exit code to indicate that the process should be restarted.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6c850b77 by Gary Lockyer at 2018-11-23T07:25:19Z
source4 smbd prefork: restart on non zero exit code

Restart any pre-fork master or worker process that exits with a non
zero exit code.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
40941e98 by Gary Lockyer at 2018-11-23T07:25:19Z
source4 smbd prefork: Add backoff to process restart

Add new smbd.conf variables 'prefork backoff increment' and
'prefork maximum backoff' to control the rate at which failed pre-forked
processes are restarted.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
11d424e1 by Gary Lockyer at 2018-11-23T07:25:20Z
source4 messaging: clean up terminated processes

Now that the smbd pre-fork process model restarts failed processes rather than
terminating, we end up with names registered to defunct processes.
This patch adds a function to clean up all the names registered to a process.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
46b164de by Gary Lockyer at 2018-11-23T07:25:20Z
source4 smbd prefork: Cleanup messaging on restart

Clean up names registered in messaging for a terminated process.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0c10c0e1 by Gary Lockyer at 2018-11-23T07:25:20Z
source4 dcerpc_server: remove irpc_add_name

Remove the irpc_add_name from dcesrv_sock_accept, as it results in two
identical names being registered for a process.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2db38370 by Gary Lockyer at 2018-11-23T07:25:20Z
samba-tool processes: display pre-fork masters and workers

Tag prefork work processes with "(worker 0)", and sort the process list
on server name to get a consistent order.

 Service:                          PID
 --------------------------------------
 cldap_server                     15588
 ...
 ldap_server                      15584
 ldap_server(worker 0)            15627
 ldap_server(worker 1)            15630
 ldap_server(worker 2)            15632
 ldap_server(worker 3)            15634
 nbt_server                       15576
 notify-daemon                    15638
 ...
 samba                                0
 ...
 wrepl_server                     15580

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
830a6502 by Gary Lockyer at 2018-11-23T07:25:20Z
WHATSNEW: prefork restart

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fabc3c9d by Tim Beale at 2018-11-23T10:40:49Z
source4 smbd prefork: Add code comments

Add some comments to the prefork code explaining what's going on.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Nov 23 11:40:49 CET 2018 on sn-devel-144

- - - - -
d629c67d by Volker Lendecke at 2018-11-23T16:14:08Z
libads: Use dom_sid_string_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
baacc703 by Volker Lendecke at 2018-11-23T19:23:57Z
libads: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Nov 23 20:23:57 CET 2018 on sn-devel-144

- - - - -
632b3ab7 by Tim Beale at 2018-11-27T02:43:17Z
selftest: Be explicit about which testenvs use ntvfs

If a testenv didn't specify any other provision arguments, then it
defaulted to using the NTVFS file server.

This patch makes it explicit, so we just pass through "--use-ntvfs" as
extra provision args in the cases we want.

(Whether all these testenvs really need to use NTVFS or not is another
question, but at least now it's easy to see which testenvs use it).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
da933b76 by Tim Beale at 2018-11-27T02:43:17Z
selftest: Make chgdcpass's NTVFS usage more obvious

The chgdcpass testenv was not passing --use-ntvfs to the provision
command, but it was still actually using NTVFS.

The reason is the smb.conf generated by provision_raw_step1() would
always try to use the s4/NTVFS file server. Because the smb.conf already
existed, this trumped what was passed to the provision command.

This patch doesn't change the chgdcpass file server. It just makes it
more obvious that chgdcpass is using NTVFS.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bfb138e9 by Tim Beale at 2018-11-27T02:43:17Z
selftest: Make testenv NTVFS usage match --use-ntvfs

Regardless of whether the testenv uses --use-ntvfs as part of its
provision options, the s4 testenvs all default to using the NTVFS file
server.

It's not particularly obvious that this is happening. The new restore
DCS (restoredc, renamedc, labdc) were all using NTVFS unintentionally.

The problem is the s4 testenvs default to using services '-s3fs +smb".
provision_ad_dc() explicitly overrides this to use s3fs again
(technically it ends up with both in its smb.conf and just uses whatever
comes last).

This patch changes the testenv setup to check for the presence of the
'--use-ntvfs' option and to set the 'server services' config option
appropriately. This way, the provision command and the smb.conf options
should always line up, with respect to NTVFS.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
88ccd418 by Tim Beale at 2018-11-27T02:43:17Z
tests: Use s3 smbclient for testenv smoketests

smbclient4 doesn't support SMBv2 connections, i.e. it won't work against
a DC with SMBv1 disabled. The smoke-test here is that the DC accepts
the connection, so we don't really care what SMB tool we use to connect.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
160031f7 by Tim Beale at 2018-11-27T02:43:17Z
selftest: Designate one testenv as having SMBv1 disabled

We recommend users disable SMBv1 to avoid potential security holes.
However, none of the AD DC testenvs have SMBv1 disabled.

This patch disables SMBv1 on an arbitrarily-chosen testenv (restoredc).

I chose restoredc as we'll want to run the backup tool tests against
this target, and it might be useful to check we can backup a DC if it's
already been restored once.

Note that SMBv2 doesn't support POSIX extensions (only SMBv1 does),
which is why we haven't just disabled SMBv1 on *all* testenvs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
93c2a952 by Tim Beale at 2018-11-27T02:43:17Z
tests: Rework backup test_backup_invalid_args test-case

self.create_backup() uses self.run_cmd(), which is a wrapper around
self.check_output(). Rework the code to call the underlying
check_output() function directly instead.

The reason we're doing this is we want run_cmd() to catch exceptions and
fail the test (i.e. in the next patch). However, we can't do that because
this test case relies on receiving the exceptions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
70dee4c1 by Tim Beale at 2018-11-27T02:43:17Z
tests: Handle backup command exceptions as test failures, not errors

If the backup command fails (i.e. throws an exception), we want the test
to fail. This makes it easier to mark tests as 'knownfail' (because we
can't knownfail test errors).

In theory, this should just involve updating run_cmd() to catch any
exceptions from the command and then call self.fail().

However, if the backup command fails, it can leave behind files in the
targetdir. Partly this is intentional, as these files may provide clues
to users as to why the command failed. However, in selftest, it causes
the TestCaseInTempDir._remove_tempdir() assertion to fire. Because this
assert actually gets run as part of the teardown, the assertion gets
treated as an error rather than a failure (and so we can't knownfail the
backup tests). To get around this, we remove any files in the tempdir
prior to calling self.fail().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2e235bda by Noel Power at 2018-11-27T02:43:17Z
python/samba/test: PY3 port samba.tests.domain_backup

The restoredc already runs under python3, so before we can run the
domain_backup tests against the restoredc, we need to make sure they
work under python3.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Noel Power <noel.power at suse.com>
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
738418ea by Tim Beale at 2018-11-27T02:43:17Z
tests: Run backup tests against restoredc (SMBv1 disabled)

Running the backup tests against the restoredc highlights that the
backup online/rename commands don't work if SMBv1 is disabled. Note that
the offline commands still work because they don't rely on an SMB
connection to the server.

(Note that running the backup tests against the restoredc is probably a
good idea anyway, to prove that there's no limit to the number of times
you can restore a domain from backup, i.e. we support more than just a
one-off restore).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cbcd8eec by Tim Beale at 2018-11-27T02:43:17Z
tests: Work out DOMSID via samdb rather than environs

Not all testenvs have the DOMSID set as an environment variable.
However, it's easy enough to work out from querying the samdb.

This is a slight change in that we use a source4-generated loadparm
to connect to the DB (self.lp is source3-generated, presumably for
some SMB connection dependency).

This change is so we can run the ntacls_backup tests against a DC with
SMBv1 disabled (the restoredc). Note that currently the tests fail in
the smb.SMB() connection in the setUp(), so we can't run them as part
of autobuild just yet (because we can't known-fail test errors).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0d15c2be by Tim Beale at 2018-11-27T05:57:03Z
tests: Rework backup test inheritance to make LP constraints clearer

The backup tests have a special constraint where we always want to use
check_output() over runcmd(). The reason is we need the samba-tool
backup/restore commands executed in a separate process. Otherwise the
global underlying LoadParm can accumulate settings from earlier test
case runs.

We can avoid someone in future inadvertently running runcmd() by
mistake, by simply changing the inheritance so we no longer inherit from
SambaToolCmdTest (so the runcmd functions are no longer present).

The comment explaining this has been moved to the top of the file.

Note that the TestCaseInTempDir inheritance was redundant.
BlackboxTestCase inherits from TestCaseInTempDir (and SambaToolCmdTest
was inheriting from BlackboxTestCase).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Tue Nov 27 06:57:03 CET 2018 on sn-devel-144

- - - - -
55a82f90 by Ralph Boehme at 2018-11-27T06:13:12Z
vfs_error_inject: add pwrite

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
523a9b31 by Ralph Boehme at 2018-11-27T06:13:13Z
vfs_error_inject: add EBADF error

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
48ddb87a by Ralph Boehme at 2018-11-27T06:13:13Z
s4:torture: add a test-suite for VSS

This test will not be run from the main torture test runner in selftest,
as there we don't pass the required arguments 'twrp_file' and
'twrp_snapshot'.

The test needs a carefully prepared environment with provisioned
snapshot data, so the test will be started from a blackbox test
script. That comes next.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
12778f01 by Ralph Boehme at 2018-11-27T06:13:13Z
s3:script/tests: add a test for VSS write behaviour

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
87bf06ed by Ralph Boehme at 2018-11-27T06:13:13Z
vfs_shadow_copy2: add _already_converted arg to shadow_copy2_strip_snapshot_internal()

Not used for now, all existing callers pass NULL.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
14d6488d by Ralph Boehme at 2018-11-27T06:13:13Z
vfs_shadow_copy2: add shadow_copy2_strip_snapshot_converted

Can be used by callers to determine if a path is in fact pointing at a
file in a snapshot. Will be used in the next commit.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
cf957562 by Ralph Boehme at 2018-11-27T06:13:13Z
vfs_shadow_copy2: nicely deal with attempts to open previous version for writing

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13688

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
cfffa2e2 by Ralph Boehme at 2018-11-27T06:13:13Z
s3:selftest: add a VSS test reading a stream

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
bffc540b by Ralph Boehme at 2018-11-27T06:13:13Z
s3:smbd: prepare filename_convert_internal() for twrp

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c69bd336 by Ralph Boehme at 2018-11-27T06:13:13Z
s3:smbd: add twrp processing to filename_convert_internal()

Not used for now, existing callers pass NULL.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
14b6e684 by Ralph Boehme at 2018-11-27T06:13:13Z
s3:smbd: add twrp args to filename_convert()

All existing callers pass NULL, no change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9c462e1b by Ralph Boehme at 2018-11-27T06:13:13Z
s3:smbd: pass down twrp from SMB2_CREATE to filename_convert()

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
aa1fac69 by Ralph Boehme at 2018-11-27T06:13:14Z
vfs_shadow_copy2: in fstat also convert fsp->fsp_name and fsp->base_fsp->fsp_name

Stacked VFS modules might use the file name, not the file
handle. Looking at you, vfs_fruit...

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13455

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f2e939b6 by Volker Lendecke at 2018-11-27T06:13:14Z
libads: Give krb5_errs.c its own header

The protos were declared in lib/krb5_wrap but the functions are not
available there.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1de97717 by Volker Lendecke at 2018-11-27T06:13:14Z
libgpo: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1d548457 by Volker Lendecke at 2018-11-27T06:13:14Z
libgpo: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d2b2d897 by Volker Lendecke at 2018-11-27T06:13:14Z
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f1446533 by Volker Lendecke at 2018-11-27T06:13:14Z
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a1ca40e6 by Volker Lendecke at 2018-11-27T06:13:14Z
winbind: Use dom_sid_str_buf

Also fix a DBG format string specifier

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e168a673 by Volker Lendecke at 2018-11-27T06:13:14Z
winbind: Fix "wbint_Principals" definition

A signed integer does not make any sense for an IDL array length

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8be87034 by Volker Lendecke at 2018-11-27T06:13:14Z
idmap_cache: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b289d15b by Volker Lendecke at 2018-11-27T06:13:14Z
libcli: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0d5287fb by Volker Lendecke at 2018-11-27T06:13:14Z
libcli: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
bb7c3b7c by Volker Lendecke at 2018-11-27T09:38:11Z
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Tue Nov 27 10:38:11 CET 2018 on sn-devel-144

- - - - -
1f42e62e by Volker Lendecke at 2018-11-27T20:42:24Z
notifyd: Improve a debug message

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Nov 27 21:42:24 CET 2018 on sn-devel-144

- - - - -
c3f60859 by Aaron Haslett at 2018-11-28T07:22:23Z
dns: prevent self-referencing CNAME

Stops the user from adding a self-referencing CNAME over RPC, which is an easy
mistake to make with samba-tool.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13600

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>

- - - - -
97b426ba by Aaron Haslett at 2018-11-28T07:22:23Z
CVE-2018-14629 dns: CNAME loop prevention using counter

Count number of answers generated by internal DNS query routine and stop at
20 to match Microsoft's loop prevention mechanism.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13600

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>

- - - - -
b6e9c4b8 by Andrew Bartlett at 2018-11-28T07:22:23Z
CVE-2018-16841 heimdal: Fix segfault on PKINIT with mis-matching principal

In Heimdal KRB5_KDC_ERR_CLIENT_NAME_MISMATCH is an enum, so we tried to double-free
mem_ctx.

This was introduced in 9a0263a7c316112caf0265237bfb2cfb3a3d370d for the
MIT KDC effort.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13628

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c835e27a by Andrew Bartlett at 2018-11-28T07:22:24Z
CVE-2018-16841 selftest: Check for mismatching principal in certficate compared with principal in AS-REQ

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13628
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
d2c98abd by Garming Sam at 2018-11-28T07:22:24Z
CVE-2018-16851 ldap_server: Check ret before manipulating blob

In the case of hitting the talloc ~256MB limit, this causes a crash in
the server.

Note that you would actually need to load >256MB of data into the LDAP.
Although there is some generated/hidden data which would help you reach that
limit (descriptors and RMD blobs).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13674

Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c1d4033e by Gary Lockyer at 2018-11-28T07:22:24Z
CVE-2018-16852 dcerpc dnsserver: Verification tests

Tests to verify
Bug 13669 - (CVE-2018-16852) NULL
            pointer de-reference in Samba AD DC DNS management

The presence of the ZONE_MASTER_SERVERS property or the
ZONE_SCAVENGING_SERVERS property in a zone record causes the server to
follow a null pointer and terminate.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13669

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2b00f8fa by Gary Lockyer at 2018-11-28T07:22:24Z
CVE-2018-16852 dcerpc dnsserver: Ensure properties are handled correctly

Fixes for
Bug 13669 - (CVE-2018-16852) NULL
            pointer de-reference in Samba AD DC DNS management

The presence of the ZONE_MASTER_SERVERS property or the
ZONE_SCAVENGING_SERVERS property in a zone record causes the server to
follow a null pointer and terminate.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13669

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9a12a001 by Gary Lockyer at 2018-11-28T07:22:24Z
CVE-2018-16852 dcerpc dnsserver: refactor common properties handling

dnsserver_common.c and dnsutils.c both share similar code to process
zone properties.  This patch extracts the common code and moves it to
dnsserver_common.c.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13669

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
07c49d25 by Andrew Bartlett at 2018-11-28T07:22:24Z
CVE-2018-16853 build: The Samba AD DC, when build with MIT Kerberos is experimental

This matches https://wiki.samba.org/index.php/Running_a_Samba_AD_DC_with_MIT_Kerberos_KDC

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13678

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c5370a43 by Andrew Bartlett at 2018-11-28T07:22:24Z
CVE-2018-16853 WHATSNEW: The Samba AD DC, when build with MIT Kerberos is experimental

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13678

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
bf0e9041 by Isaac Boukris at 2018-11-28T07:22:24Z
CVE-2018-16853: Fix kinit test on system lacking ldbsearch

By fixing bindir variable name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13571

Signed-off-by: Isaac Boukris <iboukris at gmail.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
6c453aeb by Isaac Boukris at 2018-11-28T07:22:24Z
CVE-2018-16853: The ticket in check_policy_as can actually be a TGS

This happens when we are called from S4U2Self flow, and in that case
kdcreq->client is NULL.  Use the name from client entry instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13571

Signed-off-by: Isaac Boukris <iboukris at gmail.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
c556ac5c by Isaac Boukris at 2018-11-28T07:22:24Z
CVE-2018-16853: Add a test to verify s4u2self doesn't crash

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13571

Signed-off-by: Isaac Boukris <iboukris at gmail.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
7cddbcf0 by Andreas Schneider at 2018-11-28T07:22:24Z
CVE-2018-16853: Do not segfault if client is not set

This can be triggered with FAST but we don't support this yet.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13571

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
6ab51b2a by Isaac Boukris at 2018-11-28T07:22:25Z
CVE-2018-16853: fix crash in expired passowrd case

When calling encode_krb5_padata_sequence() make sure to
pass a null terminated array as required.

Fixes expired passowrd case in samba4.blackbox.kinit test.

Signed-off-by: Isaac Boukris <iboukris at gmail.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
77de8278 by Tim Beale at 2018-11-28T07:22:25Z
CVE-2018-16857 tests: Sanity-check password lockout works with default values

Sanity-check that when we use the default lockOutObservationWindow that
user lockout actually works.

The easiest way to do this is to reuse the _test_login_lockout()
test-case, but stop at the point where we wait for the lockout duration
to expire (because we don't want the test to wait 30 mins).

This highlights a problem currently where the default values don't work.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c7b937c5 by Tim Beale at 2018-11-28T07:22:25Z
CVE-2018-16857 dsdb/util: Correctly treat lockOutObservationWindow as 64-bit int

Commit 442a38c918ae1666b35 refactored some code into a new
get_lockout_observation_window() function. However, in moving the code,
an ldb_msg_find_attr_as_int64() inadvertently got converted to a
ldb_msg_find_attr_as_int().

ldb_msg_find_attr_as_int() will only work for values up to -2147483648
(about 3.5 minutes in MS timestamp form). Unfortunately, the automated
tests used a low enough timeout that they still worked, however,
password lockout would not work with the Samba default settings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
13014aea by Tim Beale at 2018-11-28T07:22:25Z
CVE-2018-16857 dsdb/util: Fix lockOutObservationWindow for PSOs

Fix a remaining place where we were trying to read the
msDS-LockoutObservationWindow as an int instead of an int64.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fde9f7c8 by Tim Beale at 2018-11-28T10:31:14Z
CVE-2018-16857 dsdb/util: Add better default lockOutObservationWindow

Clearly the lockOutObservationWindow value is important, and using a
default value of zero doesn't work very well.

This patch adds a better default value (the domain default setting of 30
minutes).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13683

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(master): Wed Nov 28 11:31:14 CET 2018 on sn-devel-144

- - - - -
f8e24596 by Ralph Boehme at 2018-11-28T11:59:27Z
tfork/test: ensure all threads start with SIGCHLD unblocked

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
b6cd7f8c by Ralph Boehme at 2018-11-28T11:59:27Z
tfork: TFORK_ANNOTATE_BENIGN_RACE

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
b92d7905 by Ralph Boehme at 2018-11-28T11:59:28Z
tfork: add a suppresssions file for helgrind

tfork_atexit_unknown[1|2]:

  No idea what triggers this, definitely not tfork itself.

tfork_pthread_get_specific:

 Helgrind reports:

 Possible data race during read of size 4 at 0x5141304 by thread #3
 Locks held: none
    at 0x50E602E: tfork_global_get (tfork.c:301)
    by 0x50E69B1: tfork_create (tfork.c:737)
    by 0x2F7419: tfork_thread (tfork.c:431)
    by 0x4C35AC5: mythread_wrapper (hg_intercepts.c:389)
    by 0x8D38593: start_thread (pthread_create.c:463)
    by 0x9A70E6E: clone (clone.S:95)

 This conflicts with a previous write of size 4 by thread #2
 Locks held: none
    at 0x8D3F7B7: pthread_key_create (pthread_key_create.c:41)
    by 0x50E5F79: tfork_global_initialize (tfork.c:280)
    by 0x8D3FEA6: __pthread_once_slow (pthread_once.c:116)
    by 0x50E6999: tfork_create (tfork.c:728)
    by 0x2F7419: tfork_thread (tfork.c:431)
    by 0x4C35AC5: mythread_wrapper (hg_intercepts.c:389)
    by 0x8D38593: start_thread (pthread_create.c:463)
    by 0x9A70E6E: clone (clone.S:95)
  Location 0x5141304 is 0 bytes inside global var "tfork_global_key"
  declared at tfork.c:122

  This is nonsense, tfork_global_get() calls pthread_getspecific, so
  we're looking at the pthread_key_create()/pthread_[g|s]etspecific()
  API here which works with threads by design.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0a0daeba by Ralph Boehme at 2018-11-28T11:59:28Z
tfork: add a suppresssions file for drd

drd reports:

 initialized twice: cond 0x514f188
    at 0x4C3A399: pthread_cond_init_intercept (drd_pthread_intercepts.c:1022)
    by 0x4C3A399: pthread_cond_init@* (drd_pthread_intercepts.c:1030)
    by 0x50F3FF3: tfork_atfork_child (tfork.c:250)
    by 0x9A4B95D: fork (fork.c:204)
    by 0x50F4834: tfork_start_waiter_and_worker (tfork.c:581)
    by 0x50F4CDB: tfork_create (tfork.c:780)
    by 0x2F7469: tfork_thread (tfork.c:431)
    by 0x4C358F8: vgDrd_thread_wrapper (drd_pthread_intercepts.c:444)
    by 0x8D46593: start_thread (pthread_create.c:463)
    by 0x9A7EE6E: clone (clone.S:95)
 cond 0x514f188 was first observed at:
    at 0x4C3A399: pthread_cond_init_intercept (drd_pthread_intercepts.c:1022)
    by 0x4C3A399: pthread_cond_init@* (drd_pthread_intercepts.c:1030)
    by 0x50F413A: tfork_global_initialize (tfork.c:287)
    by 0x8D4DEA6: __pthread_once_slow (pthread_once.c:116)
    by 0x4C377FD: pthread_once_intercept (drd_pthread_intercepts.c:800)
    by 0x4C377FD: pthread_once (drd_pthread_intercepts.c:806)
    by 0x50F4C0E: tfork_create (tfork.c:743)
    by 0x2F7469: tfork_thread (tfork.c:431)
    by 0x4C358F8: vgDrd_thread_wrapper (drd_pthread_intercepts.c:444)
    by 0x8D46593: start_thread (pthread_create.c:463)
    by 0x9A7EE6E: clone (clone.S:95)

This is intentional, the reinit is in a child process. Cf the comment in
tfork.c.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8bde5ea1 by Ralph Boehme at 2018-11-28T14:57:43Z
tfork: add a README how to run test torture test under valgrind

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Wed Nov 28 15:57:43 CET 2018 on sn-devel-144

- - - - -
f43ac073 by Volker Lendecke at 2018-11-28T16:44:14Z
credentials: Only do shallow copies of valid ccaches

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c59f5f29 by Volker Lendecke at 2018-11-28T16:44:14Z
credentials: Fix an error path memleak

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
730beac1 by Volker Lendecke at 2018-11-28T16:44:14Z
credentials: Fix set_ccache with empty creds cache

This is an extension of bb2f7e3aee7e9b8: Without this fix in the
"empty ccache" case we never set cred->ccache, so the whole call to
cli_credentials_set_ccache became pointless

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c39526eb by Volker Lendecke at 2018-11-28T16:44:14Z
credentials: Remove an unnecessary talloc_steal()

ccc was already allocated off cred, this talloc_steal was a no-op.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a1670145 by Volker Lendecke at 2018-11-28T16:44:15Z
krb5_wrap: Add a talloc_ctx to smb_krb5_principal_get_realm()

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
899bbb44 by Volker Lendecke at 2018-11-28T16:44:15Z
lib: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
130e034f by Volker Lendecke at 2018-11-28T16:44:15Z
auth: Align integer types

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3be9ffe4 by Volker Lendecke at 2018-11-28T20:15:31Z
krb5_wrap: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Nov 28 21:15:31 CET 2018 on sn-devel-144

- - - - -
63a48e64 by Andreas Schneider at 2018-11-28T22:19:20Z
lib:replace: Check if HAVE_DECL_ENVIRON is defined first

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
9f28d8e8 by Andreas Schneider at 2018-11-28T22:19:21Z
s3: Remove unsused MMAP_BLACKLIST ifdef checks

This doesn't get defined by anything.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
bebab076 by Andreas Schneider at 2018-11-28T22:19:21Z
wafsamba: Do not always set _FORTIFY_SOURCE=2

This requires to be compiled with optimization (-O).

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
a9477eb3 by Andreas Schneider at 2018-11-28T22:19:21Z
autobuild: Add _FORTIFY_SOURCE=2 to the -O3 build

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
ab5f26f3 by Andreas Schneider at 2018-11-28T22:19:21Z
ctdb: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
ded788bc by Andreas Schneider at 2018-11-28T22:19:21Z
krb5_wrap: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
9f2cd92c by Andreas Schneider at 2018-11-28T22:19:21Z
replace: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4e8e1720 by Andreas Schneider at 2018-11-28T22:19:21Z
tdb: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
80750253 by Andreas Schneider at 2018-11-28T22:19:22Z
lib:util: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
e7f0b6e4 by Andreas Schneider at 2018-11-28T22:19:22Z
lib:util: Avoid name confusion with config.h

The HAVE_* is normally used for config.h definitions, so rename it to
USE_ASM_BYTEORDER.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
aee760cd by Andreas Schneider at 2018-11-28T22:19:22Z
libcli:smb: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
b51e8b98 by Andreas Schneider at 2018-11-28T22:19:22Z
libcli:smbreadline: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
d035f249 by Andreas Schneider at 2018-11-28T22:19:22Z
krb5_plugin: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c56b1690 by Andreas Schneider at 2018-11-28T22:19:22Z
wbclient: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4dd2349c by Andreas Schneider at 2018-11-28T22:19:22Z
wins: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
0433576d by Andreas Schneider at 2018-11-28T22:19:22Z
nss_winbind: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
be04480e by Andreas Schneider at 2018-11-28T22:19:22Z
s3:auth: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
99a9b092 by Andreas Schneider at 2018-11-28T22:19:23Z
s3:ldap: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
f4694b61 by Andreas Schneider at 2018-11-28T22:19:23Z
s3:include: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
72c6cd2d by Andreas Schneider at 2018-11-28T22:19:23Z
s3:lib: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3f3cc42b by Andreas Schneider at 2018-11-28T22:19:23Z
s3:libads: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
77b88fd0 by Andreas Schneider at 2018-11-28T22:19:23Z
s3:libsmb: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
32fda4fc by Andreas Schneider at 2018-11-28T22:19:23Z
s3:modules: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
0c69078f by Andreas Schneider at 2018-11-28T22:19:23Z
s3:nmbd: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
a7e10f78 by Andreas Schneider at 2018-11-28T22:19:23Z
s3:rpc_server: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
95432e30 by Andreas Schneider at 2018-11-28T22:19:24Z
s3:smbd: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
fce92606 by Andreas Schneider at 2018-11-28T22:19:24Z
s3:utils: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3872d4ab by Andreas Schneider at 2018-11-28T22:19:24Z
s3:winbindd: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
788d9f31 by Andreas Schneider at 2018-11-28T22:19:24Z
s4:lib: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4c4d588f by Andreas Schneider at 2018-11-28T22:19:24Z
s4:ntvfs: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
342c3b7d by Andreas Schneider at 2018-11-28T22:19:24Z
s4:smbd: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
dbbf978a by Andreas Schneider at 2018-11-29T01:20:48Z
s4:torture: Use #ifdef instead of #if for config.h definitions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Thu Nov 29 02:20:48 CET 2018 on sn-devel-144

- - - - -
e9940223 by Anoop C S at 2018-11-29T10:52:22Z
s3/testparm: Reduce debug level to 1

Adhere to what we document in manual page for testparm that default
debug level is set to reasonable value 1.

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Thu Nov 29 11:52:22 CET 2018 on sn-devel-144

- - - - -
1cf84b6b by Olly Betts at 2018-11-30T02:35:13Z
net: Add support for a credentials file

Add support for the same -A authfile/--authentication-file authfile
option that most of the other tools already do.

Signed-off-by: Olly Betts <olly at survex.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
669249ae by Olly Betts at 2018-11-30T02:35:13Z
New testcase samba3.blackbox.net_rpc_join_creds

Tests that you can now use a credentials file with net.

Signed-off-by: Olly Betts <olly at survex.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
28aeb86a by Olly Betts at 2018-11-30T02:35:13Z
Fix spelling mistakes

Signed-off-by: Olly Betts <olly at survex.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6af59922 by Olly Betts at 2018-11-30T02:35:13Z
net rpc share allowedusers: Allow restricting shares

The help already implies that you can specify "targets" for net rpc
share allowedusers, but actually the tail end of the command line
is just ignored.

This patch allows a list of shares to be specified, and only those
shares are checked, which can be much faster if you're only interested
in a few shares on a server which exports lots.

This subcommand already accepts an optional filename for the output
of net usersidlist, with a default of stdin.  Typically you'd just pipe
one command to the other so stdin is most likely what you want.  This
patch adds support for a filename of "-" to mean stdin so that you can
specify stdin explicitly when you provide a list of shares, since in
this case the filename can't be omitted.

Signed-off-by: Olly Betts <olly at survex.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3d9ba1b7 by Olly Betts at 2018-11-30T02:35:13Z
Fix net rpc share allowedusers short description

This command allows one to list allowed users, not modify them.

Signed-off-by: Olly Betts <olly at survex.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
06061d39 by Olly Betts at 2018-11-30T02:35:13Z
Add simple tests for net rpc share allowedusers

Signed-off-by: Olly Betts <olly at survex.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f03392a0 by Daniel Southward-Ellis at 2018-11-30T06:07:36Z
Converted README to markdown

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis at catalyst.net.nz>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Nov 30 07:07:36 CET 2018 on sn-devel-144

- - - - -
d8b32807 by Martin Schwenke at 2018-11-30T07:12:31Z
util: Fix include file order

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b3490049 by Andreas Schneider at 2018-11-30T10:41:44Z
replace: Correctly check for 'extern char **environ' in unistd.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Nov 30 11:41:44 CET 2018 on sn-devel-144

- - - - -
2381b4ff by Gary Lockyer at 2018-11-30T10:42:44Z
s4 smbd standard tests: limit forked processes

Tests to confirm the standard process model honours the smbd.conf
variable "max smbd processes", when forking a new process on accept.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f90cf499 by Gary Lockyer at 2018-11-30T10:42:44Z
s4 smdb standard: Limit processes forked on accept.

Limit the number of processes started by the standard model on accept.
For those services that support fork on accept, the standard model forks
a new process for each new connection. This patch limits the number of
processes to the value specified in 'max smbd processes', a value of
zero indicates that there is no limit on the number of processes that
can be forked.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5b627edc by Gary Lockyer at 2018-11-30T14:05:04Z
WHATSNEW: standard process limits

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Nov 30 15:05:04 CET 2018 on sn-devel-144

- - - - -
46a6c6ff by Ralph Boehme at 2018-12-02T06:52:34Z
vfs_fruit: avoid dereferencing fsp->base_fsp in fruit_fstat_meta_stream()

This helps avoiding a NULL dereference on systems where additional
patches modify the following condition in open_file()

  if ((open_access_mask & (FILE_READ_DATA|FILE_WRITE_DATA|FILE_APPEND_DATA|FILE_EXECUTE)) ||
      (!file_existed && (local_flags & O_CREAT)) ||
      ((local_flags & O_TRUNC) == O_TRUNC) ) {

to

  if ((open_access_mask & (FILE_READ_DATA|FILE_WRITE_DATA|FILE_APPEND_DATA|FILE_EXECUTE|DELETE_ACCESS)) ||
      (!file_existed && (local_flags & O_CREAT)) ||
      ((local_flags & O_TRUNC) == O_TRUNC) ) {

Ie addtionally check open_access_mask against DELETE_ACCESS. As a result
opens with DELETE_ACCESS go through the code that does an fd_open() plus
a subsequent fstat().

That will trigger a crash in fruit_fstat_meta_stream() when a client
wants to delete a file for deletion. When we open base file for delete,
we call open_streams_for_delete() which internally calls create-file
with NTCREATEX_OPTIONS_PRIVATE_STREAM_DELETE which prevents opening of
the base_fsp. Voila, combined with the change described above you get a
NULL deref.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sun Dec  2 07:52:34 CET 2018 on sn-devel-144

- - - - -
dd7574af by Martin Schwenke at 2018-12-03T05:56:41Z
ctdb-daemon: Exit with error if a database directory does not exist

Since 4.9.0, the log messages can be confusing if a required database
directory does not exist.  Explicitly check for database directories,
logging a clear error and exiting if one is missing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13696

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Dec  3 06:56:41 CET 2018 on sn-devel-144

- - - - -
4e9b3ed4 by Andreas Schneider at 2018-12-03T20:16:31Z
s3:lib: Fix uninitialized variable

util_tdb.c:116:7: error: ‘len’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   buf += len;
       ^~
../../source3/lib/util_tdb.c:44:6: note: ‘len’ was declared here
  int len;
      ^~~

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
86592673 by Andreas Schneider at 2018-12-03T20:16:31Z
s3:lib: Fix undefined behavior in tdb_pack()

util_tdb.c:98:5: runtime error: null pointer passed as argument 2, which
is declared to never be null

This means the second argument of memcpy() can't be NULL.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
2b2edccb by Andreas Schneider at 2018-12-03T23:23:02Z
s3:lib: Fix undefined behavior in tdb_unpack()

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Tue Dec  4 00:23:03 CET 2018 on sn-devel-144

- - - - -
14399fd8 by Aaron Haslett at 2018-12-04T04:32:09Z
CVE-2018-14629: Tests to expose regression from dns cname loop fix

These tests expose the regression described by Stefan Metzmacher in
discussion on the bugzilla paged linked below.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13600
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
34f4491d by Stefan Metzmacher at 2018-12-04T07:52:29Z
CVE-2018-14629 dns: fix CNAME loop prevention using counter regression

The loop prevention should only be done for CNAME records!

Otherwise we truncate the answer records for A, AAAA or
SRV queries, which is a bad idea if you have more than 20 DCs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13600

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Dec  4 08:52:29 CET 2018 on sn-devel-144

- - - - -
b0af0295 by Tim Beale at 2018-12-04T08:17:17Z
netcmd: Minor changes to 'group stats' command

These changes were inadvertently left off 0c910245fca70948a3.
(They were made to the 2nd patch-set iteration posted to the
mailing-list, but for some reason the first patch-set got delivered).

Changes are:
+ rework some variable names for better readability
+ Average members defaulted to int, so lost any floating point
precision.
+ Replace 'Min members' (which was fairly meaningless) with 'Median
members per group'.
+ Fix flake8 long line warnings

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d662c411 by Tim Beale at 2018-12-04T08:17:17Z
tests: Add test-case for 'group list --verbose'

Check that the number of members reported is correct.
(This change somehow got left off the ca570bd4827aa commit that was
actually delivered).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
13f57a7f by Tim Beale at 2018-12-04T08:17:17Z
traffic: Rework how assignments are generated slightly

We want to cap the number of members that can be in a group. But first,
we need to tweak how the assignment dict gets generated, so that we get
rid of the intermediary set.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5517d606 by Tim Beale at 2018-12-04T11:22:50Z
traffic_replay: Add a max-members option to cap group size

traffic_replay tries to distribute the users among the groups in a
realistic manner - some groups will have almost all users in them.
However, this becomes a problem when testing a really large database,
e.g. we may want 100K users, but no more than 5K users in each group.

This patch adds a max-member option so we can limit how big the groups
actually get.

If we detect that a group exceeds the max-members, we reset the group's
probability (of getting selected) to zero, and then recalculate the
cumulative distribution. The means that the group should no longer get
selected by generate_random_membership(). (Note we can't completely
remove the group from the list because that changes the
list-index-to-group-ID mapping).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Dec  4 12:22:50 CET 2018 on sn-devel-144

- - - - -
dfbfb7eb by Swen Schillig at 2018-12-04T17:45:38Z
waf: Utils package not defined

Fix the package name for the WafError routine.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Alexander Bokovoy <ab at samba.org>
Autobuild-Date(master): Tue Dec  4 18:45:38 CET 2018 on sn-devel-144

- - - - -
3b38dddf by Andreas Schneider at 2018-12-05T00:38:14Z
selftest: Add gooduser and eviluser to Samba3

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13699

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Böhme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
cc471448 by Andreas Schneider at 2018-12-05T00:38:14Z
s3:tests: Test for users connecting to their 'homes' share

This adds a test for CVE-2009-2813.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13699

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Böhme <slow at samba.org>

- - - - -
99695528 by Andreas Schneider at 2018-12-05T00:38:14Z
s3:smbd: Make sure we do not export "/" (root) as home dir

If "/" (root) is returned as the home directory, prevent exporting it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13699

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Böhme <slow at samba.org>

- - - - -
a92f0ccc by Andreas Schneider at 2018-12-05T04:22:43Z
s3:tests: Add test for checking that root is not allowed as home dir

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13699

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Böhme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Dec  5 05:22:43 CET 2018 on sn-devel-144

- - - - -
c46b6b11 by Ralph Boehme at 2018-12-05T07:12:16Z
selftest: test wbinfo -n and --gid-info with "NT Authority"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12164

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: David Mulder <dmulder at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
2de5f06d by Ralph Boehme at 2018-12-05T07:12:16Z
libcli/security: add dom_sid_lookup_is_predefined_domain()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12164

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: David Mulder <dmulder at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
e0f784ba by Ralph Boehme at 2018-12-05T07:12:17Z
winbindd: add some braces

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12164

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: David Mulder <dmulder at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
b512a58b by Ralph Boehme at 2018-12-05T07:12:17Z
winbindd: fix predefined domains routing in find_lookup_domain_from_sid()

Route predefined domains through the BUILTIN domain child, not passdb.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12164

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: David Mulder <dmulder at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
8b8d9fda by Ralph Boehme at 2018-12-05T10:27:22Z
winbindd: Route predefined domains through the BUILTIN domain child

Without this eg "NT Authority" didn't work:

  $ bin/wbinfo -n "NT Authority/Authenticated Users"
  failed to call wbcLookupName: WBC_ERR_DOMAIN_NOT_FOUND
  Could not lookup name NT Authority/Authenticated Users

  $ bin/wbinfo --group-info="NT Authority/Authenticated Users"
  failed to call wbcGetgrnam: WBC_ERR_DOMAIN_NOT_FOUND
  Could not get info for group NT Authority/Authenticated Users

With the patch:

  $ bin/wbinfo -n "NT Authority/Authenticated Users"
  S-1-5-11 SID_WKN_GROUP (5)

  $ bin/wbinfo --group-info="NT Authority/Authenticated Users"
  NT AUTHORITY\authenticated users:x:10002:

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12164

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: David Mulder <dmulder at suse.com>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Wed Dec  5 11:27:22 CET 2018 on sn-devel-144

- - - - -
a83e4a24 by Stefan Metzmacher at 2018-12-05T12:35:18Z
buildtools: remove unused buildtools/bin/waf-1.9

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8ba0a9a1 by Stefan Metzmacher at 2018-12-05T12:35:19Z
ctdb/wscript: use python 3.6 compatible functions

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
19d71597 by Stefan Metzmacher at 2018-12-05T12:35:19Z
wafsamba: add MODE_{744,_777}

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f87d6cbf by Stefan Metzmacher at 2018-12-05T12:35:19Z
ctdb/wscript: make use of MODE_{644,744,755,777}

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5674c21c by Daniel Southward-Ellis at 2018-12-05T15:35:33Z
Added redirect from GitHub to GitLab

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Wed Dec  5 16:35:33 CET 2018 on sn-devel-144

- - - - -
1327e2f4 by Ralph Boehme at 2018-12-06T00:43:13Z
tests:docs: reindent special_cases to one by line

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
69521c17 by Ralph Boehme at 2018-12-06T00:43:13Z
tests:docs: add a exceptions set

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ea36967c by Ralph Boehme at 2018-12-06T00:43:13Z
docs-xml: add "smbd search ask sharemode"

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ea41bf46 by Ralph Boehme at 2018-12-06T00:43:13Z
s3:smbd: use lp_smbd_search_ask_sharemode()

Parametric options have a performance impact, use the normal options
added in the previous commit.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
040a2c17 by Ralph Boehme at 2018-12-06T00:43:13Z
docs-xml: add "smbd async dosmode"

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
54b7132d by Ralph Boehme at 2018-12-06T00:43:13Z
s3:smbd: use lp_smbd_async_dosmode()

Parametric options have a performance impact, use the normal options
added in the previous commit.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
14132e89 by Ralph Boehme at 2018-12-06T00:43:13Z
docs-xml: add "smbd max async dosmode"

The parameter is added to the lists of ignored-paremteres in the
samba.docs tests, as the given default "aio max threads * 2" works only
as manpage string.

"aio max threads" can only be calculated at run time and requires a
handle to a pthreadpool_tevent which loadparm will never have.

Because of that lp_smbd_max_async_dosmode() will always return 0 as
default and it's up to the caller to calculate "aio max threads * 2" if
lp_smbd_max_async_dosmode() returns 0. Cf the next commit.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
941d7e15 by Ralph Boehme at 2018-12-06T00:43:13Z
s3:smbd: use lp_smbd_max_async_dosmode()

Parametric options have a performance impact, use the normal options
added in the previous commit.

"aio max threads" can only be calculated at run time and requires a
handle to a pthreadpool_tevent which loadparm will never have.

Because of that lp_smbd_max_async_dosmode() will always return 0 as
default and it's up to us to calculate "aio max threads * 2" if
lp_smbd_max_async_dosmode() returns 0.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
38d819e8 by Ralph Boehme at 2018-12-06T00:43:14Z
docs-xml: add "smbd getinfo ask sharemode"

Counterpart for "smbd search ask sharemode" for getinfo.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a6606d87 by Ralph Boehme at 2018-12-06T00:43:14Z
smbd: use lp_smbd_getinfo_ask_sharemode()

Counterpart for "smbd:search ask sharemode" for getinfo.

Pair-Programmed-With: Volker Lendecke <vl at samba.org>
Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9aad13cb by Ralph Boehme at 2018-12-06T04:14:03Z
WHATSNEW: document changes in SMB server parametric options

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Dec  6 05:14:03 CET 2018 on sn-devel-144

- - - - -
d9e95849 by Andreas Schneider at 2018-12-06T07:48:27Z
librpc:ndr: Fix undefined behavior in ndr_basic

librpc/ndr/ndr_basic.c:723:2: runtime error: null pointer passed as
argument 2, which is declared to never be null

The following triggered the undefined behavior:

(gdb) bt
    at librpc/gen_ndr/ndr_drsuapi.c:2318
    fn=0x7ffff6e72983 <ndr_push_drsuapi_DsReplicaObjectIdentifier3Binary>) at ../../librpc/ndr/ndr.c:1337
    at ../../source4/dsdb/schema/schema_syntax.c:2136
    drs_str=<optimized out>) at ../../source4/dsdb/schema/tests/schema_syntax.c:122
    already_setup=<optimized out>, restricted=restricted at entry=0x0) at ../../lib/torture/torture.c:442
    at ../../lib/torture/torture.c:507
    suite=0x5555563d9490, matched=0x7fffffffcef7) at ../../source4/torture/smbtorture.c:93
    matched=0x7fffffffcef7) at ../../source4/torture/smbtorture.c:95
    at ../../source4/torture/smbtorture.c:143
(gdb) f 1
1335            NDR_CHECK(ndr_push_bytes(ndr, blob.data, blob.length));
(gdb) p blob
$2 = {data = 0x0, length = 0}

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Thu Dec  6 08:48:28 CET 2018 on sn-devel-144

- - - - -
bd4bdced by Tim Beale at 2018-12-06T11:03:53Z
selftest: Don't run KCC on backup testenvs (to avoid flappiness)

KCC onthe backup domain (i.e. backupfromdc, restoredc, offlinebackupdc)
can establish new connections for replication. Depending on timing,
this can cause the join_ldapcmp test to fail, because there's an extra
object under the NTDS Settings, at the point the ldapcmp is done.

We don't need any replication to happen on the backup domain. The
backup/restore workflow in the real world should mean that the restored DC
is never run in the same network as the original DC.

This patch updates the default KCC command for the backup testenvs to be
a no-op, so the DCs won't create new connection objects.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Dec  6 12:03:53 CET 2018 on sn-devel-144

- - - - -
2f68c436 by Stefan Metzmacher at 2018-12-06T15:53:33Z
wscript_configure_system_mitkrb5: reject a system heimdal krb5-config

Review with: git show -w

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Thu Dec  6 16:53:33 CET 2018 on sn-devel-144

- - - - -
dcd68bce by Aaron Haslett at 2018-12-07T06:07:08Z
ldb: complex expression testing

Tests that prepare complex ldap expressions and equivalent python expressions,
then compare the results of the two.

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Noel Power <noel.power at suse.com>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Fri Dec  7 07:07:08 CET 2018 on sn-devel-144

- - - - -
3ef29bed by Daniel Southward-Ellis at 2018-12-07T06:08:39Z
Changed web page to webpage

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9a628c2c by Daniel Southward-Ellis at 2018-12-07T06:08:39Z
Added link to How to do Samba: Nicely

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2ba30ee2 by Daniel Southward-Ellis at 2018-12-07T06:08:39Z
Updated Website section

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
684b530a by Daniel Southward-Ellis at 2018-12-07T06:08:39Z
Changed GitHub info to GitLab

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
61068ada by Daniel Southward-Ellis at 2018-12-07T06:08:39Z
Updated "About Samba" Info in README

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d5f9bc45 by Daniel Southward-Ellis at 2018-12-07T09:12:42Z
Removed dead groups link from Mail List Etiquette

Signed-off-by: Daniel Southward-Ellis <danielsouthwardellis at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Dec  7 10:12:42 CET 2018 on sn-devel-144

- - - - -
7b60d72f by Andreas Schneider at 2018-12-07T14:33:38Z
librpc:ndr: Give the optimizer hints for ndr_push_bytes()

Also remove the redundant check in ndr_push_DATA_BLOB.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Dec  7 15:33:38 CET 2018 on sn-devel-144

- - - - -
382705f4 by Swen Schillig at 2018-12-07T18:56:16Z
ctdb: Introduce buffer.offset to avoid memmove

The memmove operation is quite expensive, therefore,
a new buffer attribute "offset" is introduced to support
an optimized buffer processing.
The optimization is to "walk" through the buffer and process
each packet until the buffer is fully processed (empty)
without requiring any memmove.
Only if a packet is in-complete, the buffer content is moved
and the new data is read from the queue.
This way almost all memmove operations are eliminated.

Signed-off-by: Swen Schillig <swen at vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
353a947b by Swen Schillig at 2018-12-07T22:27:16Z
ctdb: Adding memory pool for queue callback

The received packet is copied into a newly allocated memory chunk for further
processing by the assigned callback. Once this is done, the memory is free'd.
This is repeated for each received packet making the memory allocation / free
an expensive task. To optimize this process, a memory pool is defined which
is sized identically to the queue's buffer.
During tests it could be seen that more than 95% of all messages were sized
below the standard buffer_size of 1k.

Signed-off-by: Swen Schillig <swen at vnet.ibm.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Christof Schmitt <cs at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Fri Dec  7 23:27:16 CET 2018 on sn-devel-144

- - - - -
99a77565 by Volker Lendecke at 2018-12-07T22:29:00Z
net_rpc: Use dom_sid_equal where appropriate

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b080389f by Volker Lendecke at 2018-12-07T22:29:00Z
net_rpc: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a32564ea by Volker Lendecke at 2018-12-07T22:29:00Z
net_usershare: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
263edf7f by Volker Lendecke at 2018-12-07T22:29:00Z
winbindd_cache: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
70b94779 by Volker Lendecke at 2018-12-07T22:29:00Z
idmap_tdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
166b40b2 by Volker Lendecke at 2018-12-07T22:29:01Z
winbindd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f7f350ec by Volker Lendecke at 2018-12-07T22:29:01Z
pdb_ldap: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1b3ee68b by Volker Lendecke at 2018-12-07T22:29:01Z
net: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e5a33648 by Volker Lendecke at 2018-12-07T22:29:01Z
libnet: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
90de1431 by Volker Lendecke at 2018-12-08T01:43:48Z
rpcclient: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Dec  8 02:43:48 CET 2018 on sn-devel-144

- - - - -
81de8f06 by Christof Schmitt at 2018-12-10T01:01:14Z
ctdb: Fix hex to int conversion in h2i

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
7d271450 by Christof Schmitt at 2018-12-10T04:02:12Z
ctdb: Remove <file> parameter from pfetch usage info

The code does not implement saving the record data to a file, so update
the usage info accordingly.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Dec 10 05:02:13 CET 2018 on sn-devel-144

- - - - -
6e46d977 by Noel Power at 2018-12-10T09:38:20Z
python/samba.tests: Ensure samba-tool is called with correct python ver.

* remove unnecessary 'bin/' part of path as base BlackBox class
  will do this anyway and also ensure correct detection that
  command needs to have 'PYTHON=blah' addeded
* modify shell script so PYTHON variable if set is prepended

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
af8077e8 by Noel Power at 2018-12-10T09:38:20Z
testprogs/blackbox: make sure samba-tool is called with correct python

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
277dbd9c by Noel Power at 2018-12-10T09:38:20Z
nsswitch/tests: PY3 samba_tool call correct python

fix samba4.blackbox.rfc2307_mapping

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3519fcb9 by Noel Power at 2018-12-10T09:38:20Z
s4/utils/test: PY3 make sure we call correct python version for samba-tool

fixes samba4.blackbox.samba_tool blackbox test

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
351ca11b by Noel Power at 2018-12-10T09:38:20Z
s4/setup/tests: make sure samba-tool is called with correct py version

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1fb9887e by Noel Power at 2018-12-10T09:38:21Z
python/samba: PY3 ord needs 'str' type not int

string_to_byte_array returns not a bytearray (as the name suggests)
but a list of byte values (int). Some code expects the list so even
using a 'real' bytearray wont work.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fdf2deb3 by Noel Power at 2018-12-10T09:38:21Z
python/samba: PY3 compat py2/p3 symbol for SocketServer/socketserver

SocketServer was renamed to socketserver in Py3, this patch
create a samba.compat.SocketServer which can be used in py2 or
py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8b10c713 by Noel Power at 2018-12-10T09:38:21Z
python/samba: Py3 Use new compat.sockerserver symbol

SocketServer symbol changed in PY3 to socketserver so
we need to use a compat symbol for PY2/PY3 code.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9b18748c by Noel Power at 2018-12-10T09:38:21Z
auth/credentials: PY3 set_password should decode from unicode 'utf8'

set_password processes input using ParseTuple with "s" format, this
accepts string or unicode but...

Some py2 code is incorrectly using code like

   credentials.set_password(pass.encode('utf8'))

however that won't work in PY3. We should just make sure the string
retrieved from unicode passed in is encoded with 'utf8'
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d253e84a by Noel Power at 2018-12-10T09:38:21Z
python/samba: PY3 Credential.set_password takes string

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0d73a2b0 by Noel Power at 2018-12-10T09:38:21Z
PY3: net.change_password & net.set_password take string not bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a25bd836 by Noel Power at 2018-12-10T09:38:21Z
python/samba/tests: PY3 port samba.tests.dns

Misc hanges needed to get make test TEST=samba.tests.dns &
samb.tests.dns_fowarder to run and pass under PY3

* socket.send needs bytes not string
* rec.dwTimeStamp expects int not float (in PY3 / operator
  will give float results, for int use '//' instead)
* re.match using bytes needs a bytes search term

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
00133791 by Noel Power at 2018-12-10T09:38:21Z
PY3: wrap filter calls with list where list is expected

filter in PY2 returns list in PY3 it returns an iterator

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
679b640a by Noel Power at 2018-12-10T09:38:21Z
python/samba/tests: Py3 port for samba.tests.auth_log_netlogon_bad_creds.samba

fix unicode doesn't exist error in PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
72921682 by Noel Power at 2018-12-10T09:38:21Z
python/samba/netcmd: PY3 fix samba4.blackbox.trust_utils test

In python3 we are using ldb.bytes where we need strings
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ac7baa6e by Noel Power at 2018-12-10T09:38:21Z
s4/dsdb/tests/python: PY3 port samba4.sam.python test

Misc changes to ensure samba4.sam.python test will run under
python2/python3

* various objectSID values when formatted need to be treated
as strings for tests.

* DOMAIN_RID_USERS, DOMAIN_RID_DOMAIN_MEMBERS, DOMAIN_RID_DCS,
  DOMAIN_RID_READONLY_DCS are all integers (but attibutes are
  ldb.bytes in PY3, need to adust various assertEquals.
* Make sure password is encoded correctly

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
55a38171 by Noel Power at 2018-12-10T09:38:21Z
s4/dsdb/tests: PY3 port samba4.ldap.passwordsettings test

* Fix various assertEquals comparing ldb.bytes with string
  when running with PY3
* Fix a couple of tuple assignments to exception (not supported
  in PY3)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0cd885c1 by Noel Power at 2018-12-10T09:38:22Z
python/samba: fix default params for PY3 ConfigParser

The default params for the python3 version of the compat ConfigParser
are not correct. Code like
   foo = ConfigParser()
fails because of this.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1d635089 by Noel Power at 2018-12-10T09:38:22Z
s4/torture/gpo: Use existing code to 'run' external commands

Noticed when the smb.conf defined 'gpo update command' contained
the $PYTHON version then the exec_wait function failed to run the
command. Seems there is some issue with the arg handling. Also
there is already existing code (samba_runcmd_send) that works fine
in similar situation (e.g. when running dnsupdate etc.) so replaced
the homebrewed exec_wait functionality with the samba_runcmd util
function.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c99450db by Noel Power at 2018-12-10T09:38:22Z
python/samba: PY3 port gpo.apply smbtorture test

1) configparser.set requires string values
2) self.gp_db.store() etc. neex to pass str object for
   xml.etree.ElementTree.Element text attribute which needs
   to be text
3) tdb delete method needs bytes key
4) configparser.write needs a file opened in text mode

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
744acb27 by Noel Power at 2018-12-10T09:38:22Z
python/samba: PY3 don't call str for bytes (or str)

Note: Fix needed also for gpo.apply

minPwdAge, maxPwdAge, minPwdLength & set_pwdProperties all
have a line like

value = str(value).encode('utf8')

this is a generic type statement I guess to convert int, float etc
to utf8 encoded bytes representing the string value for those.

This worked fine in PY2 but in py3 some routine already are passing
bytes into these methods, in these cases e.g. b'200' will get converted
to "b'200'", this change only performs the conversion above for non
bytes (or str) types by replacing the above with

        if not isinstance(value, binary_type):
            value = str(value).encode('utf8')

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
750540fd by Noel Power at 2018-12-10T09:38:22Z
python/samba/kcc: PY3 fix some str versus ldb.bytes comparisons

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0cafa9bf by Noel Power at 2018-12-10T09:38:22Z
python/samba: PY3 fix failing py3 samba.tests.group_audit test

Fix bytes being compared against ldb.bytes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
991f8bfe by Noel Power at 2018-12-10T09:38:22Z
python/samba/tests: PY3 port failing samba.tests.auth_log_netlogon

Fix password encoding

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
de9dcb12 by Noel Power at 2018-12-10T09:38:22Z
python/samba/tests: PY3 port failing samba.tests.auth_log_samlogon.py

Make sure correctly encode password to utf16 and not use
unicode (which doesn't exist in PY3)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
81043cdf by Noel Power at 2018-12-10T09:38:22Z
python/samba/tests: Fix auth_log messaging problems in py3

Some tests (especially samba.tests.auth_log_netlogon_bad_creds) are
failing due to not receiving expected messages. There seems to be
some timing issue or race around the messaging bus being set up and
getting the expected events resulting from the failed netlogon.

Specifically the the order of destruction of the messaging.Messaging()
c-py objects is different under python2. Under python2 all of the
messaging.Messaging() objects are destructed *after* all the tests
are run. Note: each instance of the TestCase has it's own Messaging()
instance which is created by TestCaseXYZ.setUp, so it appears the unittest
destroys the test instances when all the tests have run whereas in
python3 we see each messaging.Messaging() instance destroyed after
each test runs.
Ok, what difference does that make ? well it seems in python3 because
each Messaging() instance is destructed after a test runs that the
associated messaging_dgm_destroy() also runs, this destroys the
global_dgm_context context which means when the next test runs the whole
messaging infrastructure needs to be built again when the next Messaging()
object is created. On the server-side this seems to result in attempts
to send messages to the listener failing first with

get_event_server: Failed to find 'auth_event' registered on the message bus to send JSON audit events to: NT_STATUS_CONNECTION_REFUSED

and subsequently with

get_event_server: Failed to find 'auth_event' registered on the message bus to send JSON audit events to: NT_STATUS_UNSUCCESSFUL

client doesn't get any more messages, test fails :-(

So, what's the difference in python2, well because the destructors for the
(4 in the case of netlogon_bad_creds) instances of Messagaging() don't run
till the end of the tests this doesn't happen and the global_dgm_context
never gets destroyed untill all the tests complete. There is some race
condition at play here, a simple sleep at the start of a failing test
fixes the problem. But... ok that isn't a possible solution here, instead
I have adjusted the base auth tests to store the Messaging() objects in a
global list forcing them to remain in scope until the tests are complete.
This ensure the behaviour is consistent across python2 & python3.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fd87c89c by Noel Power at 2018-12-10T09:38:22Z
script: Add new (temporary) pure python3 ad-dc test

Ideally we want all the tests to run under python3 by default (no
special task for this) and then convert the existing '-py3' tasks
to run the python tests with python3.
However at the moment the convertion process is not ready to do this,
for a while we need to run separate autobuild tasks for this.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c36af9c1 by Noel Power at 2018-12-10T09:38:22Z
CI: Add new CI job for new build_samba_purepy3-ad-dc job

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
89c63386 by Noel Power at 2018-12-10T09:38:22Z
CI: Remove build_samba_ad_dc_py3 CI job

We now run a purepython3 ad-dc test job, later when the whole
build is running under python3 we will resurrect build_samba_ad_dc_py3
but as (build_samba_ad_dc_py2) for python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fdf75084 by Noel Power at 2018-12-10T09:38:22Z
python/samba/test: Make sure traffic_replay is called with correct python

PY3 fix samba4.blackbox.rfc2307_mapping

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dd62f4b6 by Noel Power at 2018-12-10T09:38:23Z
python/samba/tests: PY3 make sure traffic_learner is called with correct python

Also path to traffic_learner is not in the normal 'bin' path so
also adjusted the insertion of PYTHON version to cover this

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b920d80e by Noel Power at 2018-12-10T09:38:23Z
script: PY3 port traffic_learner

Use python3 compatable print

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9db3ad9e by Noel Power at 2018-12-10T09:38:23Z
python/samba/blackbox: PY3 port for samba.tests.blackbox.traffic_learner

The order of the values in the TrafficModel is different,
but... also unfortunately output of json.dump is also
different (even when using sorted versions of the associated
dictionaries before dumping), these changes reimport the output
files into TrafficModel objects rather than comparing the actual
raw files.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4186335c by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests: Port PY3 ldap tests

Fix various assertTrue/assertEquals so ldb.bytes (PY3)
comparaisons work both in PY3 & PY2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
32c8c820 by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests/python: PY3 port samba4.tokengroups.krb5

gensec.Security.update takes bytes as param not string with py3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
60a9998e by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests: PY3 port samba4.user_account_control test

Fix dict_items are not combinable with '+' operator error, need
to convert to list first.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
42cb270b by Noel Power at 2018-12-10T09:38:23Z
lib/ldb-samba/tests: PY3 port samba4.ldap.match_rules

Various fixes

a) schema_format_value can return bytes (esp. for objectSid,
   objectGUID
b) embedded NULL now cause ValueError in py3 (was TypeError in py2)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
397ab3af by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests: PY3 fix failing samba4.ldap.notification

Convert ldb.bytes object to string for further processing

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
74aabacc by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests/python: PY3 port samba4.ldap.sites

Fix comparison of ldb.bytes with string
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c83e1a6a by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests: PY3 port samba4.ldap.vlv

Misc changes mostly around use of ldb.bytes, bytes and string to get
the test to work. Additionally results of a range cannot be combined
with a list using the '+' operator (need to enclose the range with
list).

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cc8a4eeb by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests/python: PY3 port for samba4.ldap.linked_attributes

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2b09cfe0 by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests/python: PY3 port for samba4.ldap.rodc_rwdc.python

* Make sure samba-tool is called with correct PYTHON version
* Convert ldb.bytes results to string for comparison with expected
  results

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e8413056 by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests/python: PY3 Port samba4.ldap.password_lockout

use Exception.args member as exception is no longer supports
indexing.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f8c05eb7 by Noel Power at 2018-12-10T09:38:23Z
s4/dsdb/tests/python: PY3 port samba4.deletetest

Fix misc ldb.bytes needing to be stringified for tests to succeeed

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
06ef2d51 by Noel Power at 2018-12-10T09:38:24Z
python/samba/netcmd: PY3 port for samba4.drs.samba_tool_drs_showrepl

Fix various ldb.bytes that need to be stringified in order to get
tests to pass

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d7dba6f7 by Noel Power at 2018-12-10T09:38:24Z
python/samba: PY3 port samba.tests.samba_tool.visualize_drs

* Fix calling samba-tool with correct PYTHON version
* Fix integer division needs '//' operator (this was causing
  'uncaught exception - list indices must be integers or slices,
   not float'

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c319684d by Noel Power at 2018-12-10T09:38:24Z
s4/dsdb/tests/python: partial PY3 port for samba4.ldap.sort

Test still fails after applying the following fixes

a) only decode byte or (py2) str objects
b) fix sorted function no longer use cmp func, use compat
   cmp_to_key_fn instead
c) convert ldb.bytes returned from ldb search results
d) convert sort_functions.keys() to list so array slicing works

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
34ca15fb by Noel Power at 2018-12-10T09:38:24Z
s4/dsdb/tests/python: PY3 allow test samba4.ldap.sort pass

Adjust test data to remove use of embedded NULLs.

We are getting the following exception when running the test
under python3

Exception: Exception: Traceback (most recent call last):
  File "source4/dsdb/tests/python/sort.py", line 181, in setUp
    key=cmp_to_key_fn(locale.strcoll))
ValueError: embedded null character

Looking at the source code for locale.strcoll the lhs & rhs
params are processed using 'PyUnicode_AsWideCharString(s, NULL)'

Documentation for PyUnicode_AsWideCharString states:

"Convert the Unicode object to a wide character string.
 The output string always ends with a null character. If size is
 not NULL, write the number of wide characters (excluding the
 trailing null termination character) into *size. Note that the
 resulting wchar_t string might contain null characters, which
 would cause the string to be truncated when used with most C
 functions. If size is NULL and the wchar_t* string contains null
 characters a ValueError is raised."

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
61adb75b by Noel Power at 2018-12-10T09:38:24Z
CI: Add new (TEMP) pure python3 autobuild jobs for samba-build & samba-nt4

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a522d7e0 by Noel Power at 2018-12-10T09:38:24Z
CI: Add new CI jobs for samba-purepy3 & samba-purepy3-nt4

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d01d85cc by Noel Power at 2018-12-10T09:38:24Z
CI: Remove purepy3 temporary jobs

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b209def1 by Noel Power at 2018-12-10T09:38:24Z
s4/selftest: Reenable samba.tests.gpo to run with --extra-python

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
cc11f718 by Noel Power at 2018-12-10T09:38:24Z
auth/credentials/tests: Python 3.6 avoid deepcopy error

In PY3 both deepcopy & (shallow)copy fail with

  Traceback (most recent call last):
    File "auth/credentials/tests/bind.py", line 42, in <module>
      creds_machine = copy.copy(creds)
    File "/usr/lib64/python3.6/copy.py", line 96, in copy
      rv = reductor(4)
  TypeError: can't pickle credentials.Credentials objects

This patch avoids the nasty copies but creating and populating the
Credential objects instead of copying

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9bdeb95f by Noel Power at 2018-12-10T09:38:24Z
autobuild: Remove temporary purepy3 tasks

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c316738f by Noel Power at 2018-12-10T09:38:24Z
CI: Remove the remainder py3 jobs

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a7cb10e4 by Noel Power at 2018-12-10T09:38:24Z
PY3 pure build fix some str/bytes wobblies

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
447276e2 by Noel Power at 2018-12-10T09:38:24Z
adjust ldb config for py3 build

waf code is making assumptions that main build is py2 and extra build
is py3, this results in wrong library names being used.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3eb7281f by Noel Power at 2018-12-10T09:38:25Z
buildtools/wafsamba: Decode result of Popen as unicode/string

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f4c0343d by Noel Power at 2018-12-10T09:38:25Z
selftest: Always set PYTHON to something sensible (if it isn't set)

Lots of test scripts need to run with the correct version
of python. With the correct shebang the script should run with the
correct version, the problem is that not all scripts are part
of the installation, some scripts are part of the source code,
and the shebang is not dynamically generated as yet.
It is safer if we are somewhat version neutral at the moment and
ignore the shebang and always run scripts from the test environment
with the python version (determined by PYTHON env variable) If this
env variable isn't set then set it according to the python version
that is running the tests

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
19ac50ba by Noel Power at 2018-12-10T09:38:25Z
buildtools/wafsamba: Ensure we detect the correct python.

In order to support a default python3 build we need to ensure
we detect python3 if no PYTHON env variable is set up.

Currently we detect python with

conf.find_program('python', var='PYTHON', mandatory=mandatory)

which uses PYTHON as as hint and falls back to the program name
'python' otherwise

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3e017aa7 by Noel Power at 2018-12-10T09:38:25Z
buildtools/wafsamba Provide proper pc file

To support building by default python3 we need to fix assumption
that only extra-python with py3 has valid PYTHON_SO_ABI_FLAG used to
create alt name for pc file.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a660b7fb by Noel Power at 2018-12-10T09:38:25Z
PY3: switch current build to use python3

Make sure default make and configure for all now defaults
to building with python3.

To build a samba (or sub component e.g. talloc etc.) with python3
  ./configure && make

To build a samba (or sub component e.g. talloc etc.) with python2
  PYTHON=python ./configure && PYTHON=python make

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0186f273 by Noel Power at 2018-12-10T09:38:25Z
CI: Run autobuild jobs with python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0e35d741 by Noel Power at 2018-12-10T09:38:25Z
script: Fix failing build_xc job

build_xc job uses compare_cc_results.py to compare cache
files, the cache files are stringified hash maps, the results
in python 3.4 don't compare well due to inconsistent order of
dict key/value pairs when the cache files are created. While
comparing the file contents works fine in python3.6 it fails
with python3.4. This patch detects problematic dict lines and
rewrites the value for comparison

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ae5dd181 by Noel Power at 2018-12-10T09:38:25Z
autobuild: Modify old samba_buildpy3_only job to python2

Since autobuild now builds python3 by default we need to change
the previously buildpy3 only job to python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
23db658e by Noel Power at 2018-12-10T09:38:25Z
CI: convert the purepy3 job to a py2 one

With patches to have python3 as default we need a
pure python2 build

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dfe5787c by Noel Power at 2018-12-10T09:38:25Z
autobuild: Adjust autobuild for PY2/PY3 get_python_lib behaviour

The resuls of get_python_lib are different between python2 & python3
and this results in autobuild generating the wrong PYTHONPATH with
python3.

python2
=======
print ("%s" % get_python_lib(standard_lib=1, prefix='/my/prefix'))
/my/prefix/lib64/python2.7

python3
print ("%s" % get_python_lib(standard_lib=1, prefix='/my/prefix'))
/my/prefix/lib/python3.6

But with addition of plat_specific param the results are the same

python2
=======
print ("%s" % get_python_lib(plat_specific=1, standard_lib=0, prefix='/my/prefix'))
/my/prefix/lib64/python2.7/site-packages

python3
=======
print ("%s" % get_python_lib(plat_specific=1, standard_lib=0, prefix='/my/prefix'))
/my/prefix/lib64/python3.6/site-packages

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2fb74e6a by Noel Power at 2018-12-10T09:38:25Z
buildtools/wafsamba: Support --extra-python with python2

Relax restriction on extra_python version

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e3c37bba by Noel Power at 2018-12-10T09:38:25Z
selftest: don't hardcode '.python3' for extra-python tests

Instead of hardcoding '.python3' we now hardcode the extra python
exe (which will be python2 for a default python3 build)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b0b9c62e by Noel Power at 2018-12-10T09:38:25Z
autobuild: Convert old py3 tasks to py2

Now that we are building with python3 by default we need to
convert the old python3 test tasks to python2 (e.g. reverse how
we used do it)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5a804050 by Noel Power at 2018-12-10T09:38:25Z
CI: Add new py2 CI jobs to replace old py3 ones

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
35aef220 by Noel Power at 2018-12-10T09:38:26Z
selftest/knownfail: Add python2 version of known fails

Post build & test running under python3 we now run with
 '--extra-python=/usr/bin/python2',  these tests will get
python2 appended to the test name so we need also to create
new knownfails for these. We will keep the python3 versions
in case we create (and we probably should) some CI job(s)
with
  PYTHON=python configure.developer --extra-python=/usr/bin/python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9ac432d4 by Noel Power at 2018-12-10T09:38:26Z
selftest/flapping.d: Add python2 versions for flapping tests

Post build & test running under python3 we now run with
'--extra-python=/usr/bin/python2',  these tests will get
python2 appended to the test name so we need also to create
new flapping*/* entries for these. We will keep the python3
versions in case we create some CI job(s)
with
    PYTHON=python configure.developer --extra-python=/usr/bin/python3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bc92277a by Noel Power at 2018-12-10T09:38:26Z
autobuild: We should run autobuild with python3 by default

sn-devel autobuild runing autobuild.py (via git hooks I suppose) but
if run directly (e.g. depending on script shebang) then 'python' aka
python2 will run. This will cause an error when building some targets
because the autobuild script itself sometimes builds paths based
on the version of python executing the script e.g ${PYTHON_PREFIX}.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f7516971 by Noel Power at 2018-12-10T13:32:24Z
CI: Adjust CI tasks for new python3 autobuild.py default

Now that autobuild has defaulted to python3 (via shebang) we no longer
need to explicity call autobuild.py with 'python3'

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Mon Dec 10 14:32:24 CET 2018 on sn-devel-144

- - - - -
a6805552 by Volker Lendecke at 2018-12-10T23:40:29Z
pdb_ldap: Fix typos

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2b1125fa by Volker Lendecke at 2018-12-10T23:40:30Z
passdb: dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d8334031 by Volker Lendecke at 2018-12-10T23:40:30Z
auth: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
de331ce9 by Volker Lendecke at 2018-12-10T23:40:30Z
lib: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
eaa035b8 by Volker Lendecke at 2018-12-10T23:40:30Z
net: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c48d6345 by Volker Lendecke at 2018-12-10T23:40:30Z
profiles: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
30284017 by Volker Lendecke at 2018-12-10T23:40:30Z
net: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
49f7dc01 by Volker Lendecke at 2018-12-10T23:40:30Z
lib: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d20c57c2 by Volker Lendecke at 2018-12-10T23:40:30Z
sharesec: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2c179bcd by Volker Lendecke at 2018-12-10T23:40:30Z
pdbedit: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
56afb781 by Volker Lendecke at 2018-12-10T23:40:30Z
pdb_ldap: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
38a39701 by Volker Lendecke at 2018-12-10T23:40:31Z
libnet: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b0077bb0 by Volker Lendecke at 2018-12-10T23:40:31Z
rpc_server3: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
918602e0 by Volker Lendecke at 2018-12-10T23:40:31Z
rpcclient: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
bea43789 by Volker Lendecke at 2018-12-10T23:40:31Z
torture3: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f4ee9551 by Volker Lendecke at 2018-12-10T23:40:31Z
vfs: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
da1d1e80 by Volker Lendecke at 2018-12-10T23:40:31Z
lib: Remove sid_string_tos

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5ba64d49 by Volker Lendecke at 2018-12-10T23:40:31Z
netapi: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
12f3a37a by Volker Lendecke at 2018-12-10T23:40:31Z
libads: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6e55ca9e by Volker Lendecke at 2018-12-10T23:40:31Z
net: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
939d64b9 by Volker Lendecke at 2018-12-10T23:40:31Z
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9ffa6d49 by Volker Lendecke at 2018-12-10T23:40:31Z
libgpo: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6a7a9933 by Volker Lendecke at 2018-12-11T03:22:33Z
lib: Remove sid_string_talloc

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Dec 11 04:22:33 CET 2018 on sn-devel-144

- - - - -
884eeca3 by Andreas Schneider at 2018-12-11T12:58:30Z
s4:web_server: Fix build error

source4/web_server/wsgi.c:149:8: error: assignment discards 'const'
qualifier from pointer target type [-Werror=discarded-qualifiers]
    str = PyStr_AsUTF8AndSize(item, &size);
            ^

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Tue Dec 11 13:58:30 CET 2018 on sn-devel-144

- - - - -
75d15484 by Günther Deschner at 2018-12-11T16:26:31Z
s3-vfs: Prevent NULL pointer dereference in vfs_glusterfs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13708

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Günther Deschner <gd at samba.org>
Autobuild-Date(master): Tue Dec 11 17:26:31 CET 2018 on sn-devel-144

- - - - -
923010d9 by Noel Power at 2018-12-11T19:07:18Z
various: Remove references to about to be deleted thirdparty/dnspython

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8d333f43 by Noel Power at 2018-12-11T22:37:43Z
third_party/dnspython: Remove dnspython library from third_party

This version of dnspython isn't python3 compatible and is
quite old.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Tue Dec 11 23:37:43 CET 2018 on sn-devel-144

- - - - -
25ab4293 by Douglas Bagnall at 2018-12-12T03:38:13Z
autobuild: py3: cope with bytes when compiling system-info.txt

The command output looks like b'foo\nbar' in string-space.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
35e1a798 by Douglas Bagnall at 2018-12-12T03:38:13Z
autobuild: convert top_commit_msg to str for Py 3

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fa77209d by Tim Beale at 2018-12-12T03:38:13Z
tests: Add SMB Py binding .chkpath() test case

chkpath was only tested incidentally (and that assertion was wrong). Add
a proper test to prove it works correctly. We can then clean-up the
incorrect assertion in the next patch.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b84cea18 by Tim Beale at 2018-12-12T03:38:13Z
tests: Fix SMB Py binding .unlink() test case assertion

The current assertion would never detect if the unlink API is broken.
The chkpath() API is only useful for checking if directories exist, so
it will always return False for a regular file (regardless of whether
the file actually exists or not).

Rework the test case so we assert that the file exists by trying to read
its contents (which will throw an error if the file doesn't exist).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
629a41fc by Tim Beale at 2018-12-12T03:38:13Z
tests: Extend SMB Py binding .list() test-case

Extend the tests to better reflect some of the .list() functionality we
expect.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
da30c824 by Tim Beale at 2018-12-12T03:38:13Z
tests: Extend SMB test_save_load_text case to check overwrite

Extend the test case to check overwriting a file as well. Currently this
has the behaviour of appending to the existing file, rather than
overwriting the file with new contents.

It's not clear from the API that this is the intended behaviour in this
case, so I've marked it as a failure.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
31d4f692 by Tim Beale at 2018-12-12T03:38:13Z
s4:libcli: Fix error in smbcli_deltree()

Commit 094afe614b6282 fixed an uninitialized variable, which meant we
tried to delete the file twice. The 2nd time fails, so the function
returns an error, instead of success (even though the file is now gone).

Note we want to be using the source3 SMB library code going forward.
However, fixing this bug makes it easier to write tests against the
(currently s4) SMB python bindings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bc179e6b by Tim Beale at 2018-12-12T07:23:07Z
tests: Add SMB Py binding .deltree test case

Add a more thorough test case that .deltree works as expected.

Note that we get a slightly different NT_STATUS error in file_exists()
if the parent directory doesn't exist, e.g.
/non-existent-dir/nonexistent.txt

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Dec 12 08:23:07 CET 2018 on sn-devel-144

- - - - -
9abe35ce by Andreas Schneider at 2018-12-12T08:13:23Z
s3:modules: Remove superfloues sha256.h include in vfs_acl_tdb

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0a001515 by Andreas Schneider at 2018-12-12T08:13:23Z
s3:modules: Remove superfloues sha256.h include in vfs_acl_xattr

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
55b2f247 by Andreas Schneider at 2018-12-12T08:13:23Z
s4:torture: Do not check if the alloc_size is 0 on empty files

The allocation size might not be zero. This depends on the file system
behavior and also on the size of the extended attributes stored on the
file. E.g. If a large user.DOSATTRIB xattr is stored on XFS/ext4 then 8
blocks are used and we will round up to several megabytes.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
a80fee50 by Justin Stephenson at 2018-12-12T11:51:24Z
s4:torture/smb2/session: Fix expire tests

When run with MIT kerberos, the smb2 session expire tests fail when run
against the ad_member test environment. The krb5 library initializes
values from the private krb5.conf profile
st/ad_member/lockdir/smb_krb5/krb5.conf.ADDOMAIN, this file does not
contain a defined clockskew setting. The expire tests require a low
clockskew value that is set in st/ad_member/lib/krb5.conf.

This patch disables the creation of the private krb5.conf for the
ad_member_idmap_rid testenv, and runs the smb2.session tests
against ad_member_idmap_rid instead of ad_member.

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Wed Dec 12 12:51:24 CET 2018 on sn-devel-144

- - - - -
eabe6d53 by Andreas Schneider at 2018-12-12T17:34:10Z
lib:talloc: Fix undefined behavior in talloc_memdup

lib/talloc/talloc.c:2419: runtime error: null pointer passed as argument
2, which is declared to never be null

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
e7a8e4e6 by Andreas Schneider at 2018-12-12T17:34:11Z
libcli:security: Do not duplicate invalid aces

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
9c507e98 by Andreas Schneider at 2018-12-12T21:18:52Z
libcli:security: Return early if there are no aces to duplicate

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Wed Dec 12 22:18:52 CET 2018 on sn-devel-144

- - - - -
c71417a0 by Andrew Bartlett at 2018-12-13T00:49:30Z
waf-py3: Allow waf build --dup-symbol-check to operate in python3

Use the b prefix on output from subcommands to match bytes with bytes.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
a42f18cc by Andrew Bartlett at 2018-12-13T00:49:30Z
selftest: Run waf build --dup-symbol-check under $PYTHON

This would often be python3 in the new build

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
19a4d3ca by Andrew Bartlett at 2018-12-13T00:49:30Z
build: Workaround python3 hash order issues (for now)

This works around python3 having a new hash seed each time it starts to allow
a second "make" not to rebuild the world.

This should probably be reverted once we find the hash that is causing
the issue, but should reduce frustration for now.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
676a8fde by Andreas Schneider at 2018-12-13T00:49:30Z
pidl: Use C99 initializers for last element in ndr_interface_call(_pipes)

librpc/gen_ndr/ndr_witness.c:1421:2: warning: missing initializer for
field ‘in_pipes’ of ‘const struct ndr_interface_call’
[-Wmissing-field-initializers] <--[gcc]
  { NULL, 0, NULL, NULL, NULL }
  ^

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5508c19c by Andreas Schneider at 2018-12-13T00:49:30Z
pidl: Fix function for py_<name>_ndr_print_(in|out)

The function takes two PyObject arguments, See:

https://docs.python.org/3/c-api/structures.html#c.PyCFunction

and

https://docs.python.org/2/c-api/structures.html#c.PyCFunction

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
57783d62 by Andreas Schneider at 2018-12-13T04:00:20Z
pidl: Fix unsigned integer comparison warning

bin/default/librpc/gen_ndr/py_wkssvc.c:27344:33: warning:
comparison of integer expressions of different signedness: ‘long int’
and ‘long long unsigned int’ [-Wsign-compare] <--[gcc]
    if (test_var < 0 || test_var > uint_max) {
                                 ^

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Dec 13 05:00:20 CET 2018 on sn-devel-144

- - - - -
660b8721 by Stefan Metzmacher at 2018-12-13T07:52:23Z
wafsamba: fix pidl dependencies to rebuild on pidl changes

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
59cb025e by Stefan Metzmacher at 2018-12-13T07:52:23Z
s3:pylibsmb: pass self to py_tevent_req_wait_exc()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1bccbfcf by Stefan Metzmacher at 2018-12-13T07:52:23Z
s3:pylibsmb: only use poll_mt backend if multi_threaded=True is specified

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
247a71b6 by Stefan Metzmacher at 2018-12-13T07:52:23Z
s3:pylibsmb: add sign=True to require signing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
790dab57 by Stefan Metzmacher at 2018-12-13T07:52:23Z
s3:pylibsmb: add force_smb1=True in order to control forcing of SMB1

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fdc62b03 by Stefan Metzmacher at 2018-12-13T07:52:23Z
s3:pylibsmb: remember that a connection uses SMB1

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
aaf7aaa6 by Tim Beale at 2018-12-13T07:52:23Z
s3:pylibsmb: .get_oplock_break API is dependent on multi_threaded=True

The .get_oplock_break is dependent on the pthread code, which is only
used when creating a SMB connection with multi_threaded=True.

Add an explicit error to the .get_oplock_break() if someone tries to use
it in non-multithreaded mode.

Initializing self->oplock_waiter in non-multithreaded mode is similarly
redundant if the API can never be used.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
60148c9e by Stefan Metzmacher at 2018-12-13T07:52:23Z
s3:pylibsmb: make use of PyBytes_FromStringAndSize() in py_cli_read()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0af6b335 by Stefan Metzmacher at 2018-12-13T07:52:24Z
s3:pylibsmb: make use of PYARG_BYTES_LEN in py_cli_write()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6a3d2c33 by Stefan Metzmacher at 2018-12-13T07:52:24Z
s3:libsmb: add cli_write_send/recv which work with SMB1/2/3

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9fb0d8e7 by Tim Beale at 2018-12-13T07:52:24Z
s3:libsmb: add comments for cli_write_send/cli_push_send

Added a code comment highlighting this 2 APIs do similar jobs, and tried
to explain why you might want to use one over the other.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3c3b4400 by Stefan Metzmacher at 2018-12-13T07:52:24Z
s3:pylibsmb: make use of protocol independent cli_write_send/recv in py_cli_write()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
64e68abd by Stefan Metzmacher at 2018-12-13T07:52:24Z
s3:pylibsmb: make use of protocol independent cli_read_send/recv in py_cli_read()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
be464c1d by Stefan Metzmacher at 2018-12-13T07:52:24Z
s3:libsmb: pass impersonation_level to cli_smb2_create_fnum_send()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
850aef94 by Stefan Metzmacher at 2018-12-13T07:52:24Z
s3:libsmb: pass impersonation_level to cli_smb2_create_fnum()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ed0deadf by Stefan Metzmacher at 2018-12-13T07:52:24Z
s3:libsmb: pass ImpersonationLevel to cli_ntcreate1_send()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c8a5e89d by Stefan Metzmacher at 2018-12-13T07:52:24Z
s3:libsmb: pass impersonation_level to cli_ntcreate_send()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ecafdcb3 by Stefan Metzmacher at 2018-12-13T11:35:05Z
s3:pylibsmb: allow ImpersonationLevel argument to create()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Thu Dec 13 12:35:06 CET 2018 on sn-devel-144

- - - - -
e9a13d7f by Stefan Metzmacher at 2018-12-13T23:49:31Z
selftest/knownfail.d/smb: avoid explicit python version

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Tim Beale <timbeale at samba.org>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Fri Dec 14 00:49:31 CET 2018 on sn-devel-144

- - - - -
d20f0698 by Noel Power at 2018-12-13T23:51:32Z
selftest: Convert samba4.ldap.sort.python to planoldpythontest

Currently the test only runs fully under python2. As an interim
measure while we figure out how to get this to work property under
python3 I have converted the test to planoldpythontest so it also
can run under python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2b3c9b8e by Noel Power at 2018-12-13T23:51:32Z
s4/dsdb/tests/python: Restore embed NULL tests for Python3

commit: 34ca15fb042e42773854c093ad9f1e67696c90ac changed the
test so embedded NULLs were avoided when python3 was used.
This was due to the fact the string comparison function
'locale.strcoll' cannot handle embedded NULLs. This commit

a) Restores the test data using embedded NULLs which was
   not used depending on the python runtime version
b) Removes the problematic calculation of expected sorting order
   and instead uses sort order data stored in files.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
28ecdd97 by Noel Power at 2018-12-13T23:51:32Z
buildtools/wafsamba: Decode output of cmd_output (which is bytes)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
166d119d by Noel Power at 2018-12-13T23:51:32Z
dynconfig/wscript: python3 fix string.find instance

string.find doesn't exist in python3. Instead use the 'find' method
of the string instance itself

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9116cc8c by Noel Power at 2018-12-13T23:51:33Z
s4/scripting: python3 fix string.find instance

    string.find doesn't exist in python3. Instead use the 'find' method
    of the string instance itself

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0756b59f by Noel Power at 2018-12-13T23:51:33Z
WHATSNEW: document build system default python version change.

build now uses python3 by default instead of python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bae5364b by Noel Power at 2018-12-13T23:51:33Z
buildtools/wafsamba: re-write shebang for delivered python scripts

Can't see how the orig code would have worked though
a)
  task.env["PYTHON"] is a list
b) task.env["PYTHON_SPECIFIED"] can (and is in our case false) looks
   like it would only be true for python2 but in anycase no harm we
   always rewrite the shebang

So now it works as follows,

1. PYTHON (which is where the shebang is got) is set to python3 by default
2. To override the default you need to set PYTHON (e.g. to build with python2)
3. If you give a full path in PYTHON then shebang is of the format
   "#!{FULL_PYTHON_INTERPRETER_PATH)
4. If you specify PYTHON=python or PYTHON=python2 etc. shebang format is
   "#!!/usr/bin/env python", "#!!/usr/bin/env python2" etc.

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
90f0460c by Andrew Bartlett at 2018-12-13T23:51:33Z
buildtools/wafsamba: Ensure default python picked up is python3

1) set the default python searched for samba waf to be python3
2) remove default setting of PYTHON variable if not defined (not needed)

Signed-off-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
288338b4 by amitkuma at 2018-12-13T23:51:33Z
Correction of comment in winbindd_pam.c

Internal comment corrected.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
89705734 by Brian Candler at 2018-12-13T23:51:33Z
Add #include <unistd.h> to provide prototype for getopt()

Signed-off-by: Brian Candler <b.candler at pobox.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

- - - - -
49c1c0c3 by Philipp Gesang at 2018-12-13T23:51:33Z
tests/smbcontrol: fix typo in class name

Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f183fd32 by Brian Candler at 2018-12-14T03:13:38Z
Change order of flags in Makefile so that crackcheck builds under Ubuntu

Signed-off-by: Brian Candler <b.candler at pobox.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Dec 14 04:13:38 CET 2018 on sn-devel-144

- - - - -
edab1318 by Andrew Bartlett at 2018-12-14T09:27:18Z
audit_logging: Remove debug log header and JSON Authentication: prefix

Feedback from real-world users is that they really want raw JSON
strings in the log.

We can not easily remove the leading "  " but the other strings above
and before the JSON are really annoying to strip back off

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13714

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
b776ba80 by Andrew Bartlett at 2018-12-14T09:27:18Z
lib/audit_logging: Use talloc_stackframe() in audit_log_json()

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
31957c7f by Andrew Bartlett at 2018-12-14T13:32:25Z
audit_logging: auth_json_audit required auth_json

To log JSON the human-readable logs must also have been enabled

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13715

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Dec 14 14:32:25 CET 2018 on sn-devel-144

- - - - -
5ddff307 by Andrew Bartlett at 2018-12-14T13:40:19Z
build: Move python detection back into waf (instead of in configure and Makefile)

This avoids creating a mini-configure in the configure script.

Users wishing to use python2 to build need to specify PYTHON=
to both ./configure and make

After we merged the python3 change, it became clear that relying on systems prefixing
the correct python just causes trouble and make debugging harder, so only use $PYTHON
for the override, not the default case

This essentially reverts a660b7fb8e519bd3be558fd0425bff8f287fca1f but
leaves the files more consistent.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
934e336a by Joe Guo at 2018-12-14T13:40:20Z
s3:test: make sids2xids test compatible with py2 and py3

define a wrapper function for subprocess.check_output,
to return bytes for py2 and unicode for py3, and replace Popen with it.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
a19d5bd7 by Joe Guo at 2018-12-14T13:40:20Z
PY3: change shebang to python3 in source4/scripting/bin dir

Samba default python is 3 now.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
359d832e by Joe Guo at 2018-12-14T13:40:20Z
PY3: change shebang to python3 in source4/torture dir

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
338413ad by Joe Guo at 2018-12-14T13:40:20Z
PY3: change shebang to python3 in source4/scripting dir

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
8e3c1944 by Joe Guo at 2018-12-14T13:40:20Z
PY3: change shebang to python3 in source4/dsdb dir

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
9d8e35ab by Joe Guo at 2018-12-14T13:40:20Z
PY3: change shebang to python3 in script dir

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
c65a7078 by Joe Guo at 2018-12-14T13:40:20Z
PY3: change shebang to python3 in wintest dir

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
a07279b7 by Joe Guo at 2018-12-14T13:40:20Z
PY3: change shebang to python3 in lib dir

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
edfb6cb8 by Joe Guo at 2018-12-14T17:00:40Z
PY3: change shebang to python3 in misc dirs

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Fri Dec 14 18:00:40 CET 2018 on sn-devel-144

- - - - -
cb23a034 by Gary Lockyer at 2018-12-14T17:57:52Z
idl: Add Windows event code ids

Add idl definitions for Windows Event Code Ids, and Logon Types. This
intial commit adds:

Event Ids
	4264	Successful logon
	4625	Unsuccessful logon

Logon Types
	 2	Interactive
	 3	Network
	 4	Batch
	 5	Service
	 7	Unlock
	 8	NetworkCleartext
	 9	NewCredentials
	10	RemoteInteractive
	11	CachedInteractive

The intention is to add Windows Event Codes to the JSON log messages, to
provide a common event identifier in mixed Windows and Samba networks.
And to assist security personnel with a windows background.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b7baf96b by Gary Lockyer at 2018-12-14T17:57:52Z
auth log: Add windows event codes

Add a new "eventId" element to the Authorisation JSON log messages.
This contains a Windows Event Code Id either:
	4624	Successful logon
	4625	Unsuccessful logon

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ac51f155 by Gary Lockyer at 2018-12-14T17:57:52Z
auth log: Add windows logon type codes

Add a new "logonType" element to the Authorisation JSON log messages.
This contains a Windows Logon Type, the supported logon types are:
	2	Interactive
	3	Network
	8	NetworkCleartext

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
105cdd6c by Gary Lockyer at 2018-12-14T17:57:52Z
auth log: Increment Authentication version

Update the minor version of the Authorisation to reflect the addition of
the "eventId" and "logonType" elements.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e97acc71 by Gary Lockyer at 2018-12-14T17:57:52Z
dsdb audit_log: Add windows event codes to password changes

Add a new "eventId" element to the PasswordChange JSON log messages.
This contains a Windows Event Code Id either:
	4723	Password changed
	4724	Password reset

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8cdfc7bb by Andrew Bartlett at 2018-12-14T21:04:20Z
travis-ci: Remove hooks for Travis CI (typically used with GitHub)

These will get out of date pretty quickly, so better to remove them now
that GitLab CI is the documented and supported solution.

This was our first public CI system, used because it is cost-free
to Open Source projects and integrated with GitHub.

It has served it's purpose now.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Dec 14 22:04:20 CET 2018 on sn-devel-144

- - - - -
b0cabfd2 by Andreas Schneider at 2018-12-16T20:04:09Z
replace: Fix checking for config.h #define in readline.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
dd048f91 by Andreas Schneider at 2018-12-16T20:04:09Z
replace: Fix checking for config.h #define in gssapi.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
80c4bc35 by Andreas Schneider at 2018-12-16T20:04:09Z
replace: Fix checking for config.h #define in replace.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
9005943a by Andreas Schneider at 2018-12-16T20:04:09Z
lib:util: Fix checking for config.h #define in fault.c

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
20d279fc by Andreas Schneider at 2018-12-16T20:04:09Z
krb5_samba: Fix checking for config.h #define in krb5_samba.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
648e70ba by Andreas Schneider at 2018-12-16T20:04:10Z
nsswitch: Fix checking for config.h #define in nsstest.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
7ea82eac by Andreas Schneider at 2018-12-16T20:04:10Z
nsswitch: Fix checking for config.h #define in winbind_nss.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
2f6ed306 by Andreas Schneider at 2018-12-16T20:04:10Z
s3: Fix checking for config.h #define in includes.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3da98cb1 by Andreas Schneider at 2018-12-16T20:04:10Z
s3:lib: Fix checking for config.h #define in system.c

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
8a313bbd by Andreas Schneider at 2018-12-16T23:10:10Z
lib:tsocket: Check for DOXYGEN as a #define

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Mon Dec 17 00:10:10 CET 2018 on sn-devel-144

- - - - -
ce33dd11 by Andrew Bartlett at 2018-12-17T00:24:14Z
dsdb: Remove readOnlySchema concept from Samba

This is a hold-over from the LDAP backend project, which has not yet been revived.

There will be bigger issues than what to do if the schema changes if this ever comes back
and our schema code is way to complex at the moment.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
0a449c2b by Guo Qiao at 2018-12-17T00:24:14Z
s4:scripting:bin: rm rpcclient

This file only works in py2, and no one use it. Just delete it.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
97c5a698 by Adam Nielsen at 2018-12-17T00:24:15Z
manpage: Advise vfs_fruit:veto_appledouble=yes can break rsync

Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Uri Simchoni <uri at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
440ddf84 by Marcos Mello at 2018-12-17T00:24:15Z
Send status to systemd on daemon start

systemd service files run in no-forking mode (--foreground) since
8b6f58194da7e849cdb9d20712dff49b17a93a77.

Rearrange sd_notify() call in become_daemon() to only send status to systemd
in this mode (Type=notify is not designed to monitor forking). Drop READY=0
(it does nothing) and MAINPID= (unnecessary because the process spawned by
systemd is already the main PID).

Also remove STATUS= prefix from debug messages.

Signed-off-by: Marcos Mello <marcosfrm at gmail.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f460bb52 by Will at 2018-12-17T00:24:15Z
Conditionally disable macOS incompatible tests

Symbols _getgrent_r and _getpwent_r in
source4/torture/local/nss_tests.c are undefined in macOS. It seems
that checking HAVE_GETGRENT_R and HAVE_GETPWENT_R and conditionally
disabling those tests as suggested by hirochachacha in the referenced
bug allows samba on both `master` and `samba-4.7.1` to build properly
on macOS/darwin.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11984

Signed-off-by: Will Haley <willhy at gmail.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1fd4cdfa by Aaron Haslett at 2018-12-17T00:24:15Z
drepl: schema repl race condition fix

Adds final schema consistency check before committing changes.
Aborts if corruption found.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12889
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1cff50fe by Andrew Bartlett at 2018-12-17T03:30:39Z
dsdb: sort DSDB_EXTENDED defines by OID

This helps avoid duplicate values and clearly indicates what value to select next.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Dec 17 04:30:39 CET 2018 on sn-devel-144

- - - - -
b77fae32 by Volker Lendecke at 2018-12-18T01:00:44Z
smbd: Don't try to release a kernel oplock for a leased file

If we have

[global]
  smb2 leases = yes
  kernel oplocks = no
[share]
  kernel oplocks = yes

for clients requesting leases we don't even try to acquire kernel
oplocks, because the kernel API is not compatible. Kernel oplocks are
per fd, leases are roughly "per inode".

We don't however special-case the LEASE_OPLOCK case in
release_file_oplock, leading to nasty error messages like "bad file
descriptor" on the fcntl(fd,F_SETLEASE,F_UNLCK) call. They are
harmless, but they raise eyebrows.

To simplify the if-condition, I factored out the kernel call and
applied early returns.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Dec 18 02:00:44 CET 2018 on sn-devel-144

- - - - -
93284ed0 by Martin Schwenke at 2018-12-18T01:02:03Z
ctdb-daemon: Divide by 2 when calculating hop count bucket

This provides finer resolution while still maintaining a reasonable
maximum.  In this case the top bucket contains any hop counts
>= 16384, compared to the current situation where the top bucket contains
hop counts >= 268435456.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
9d1d5fa4 by Martin Schwenke at 2018-12-18T01:02:03Z
ctdb-doc: Add non-breaking space to lock_buckets documentation

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
da8aaf2a by Martin Schwenke at 2018-12-18T01:02:03Z
ctdb-recoverd: Call an election when the recovery lock is lost

The lock may have been lost due to a failure in the underlying locking
mechanism.  This could be due to quorum loss or similar.  It is best
to call an election to confirm that this node should still be master.
At worst, the node will reelect itself, fail to take the lock and then
ban itself.  This is a suitable outcome for a node that has been
partitioned from others in the cluster.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
ba83aa9d by Martin Schwenke at 2018-12-18T01:02:03Z
ctdb-event: Force script directory to be absolute

If TEST_VAR_DIR is relative then symbolic link targets will be
incorrect.

Don't force TEST_VAR_DIR to be absolute because this can result in
Unix domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
45f96c73 by Martin Schwenke at 2018-12-18T01:02:04Z
ctdb-event: Force EVENTSCRIPTS_TESTS_VAR_DIR to be absolute

Event scripts (well, statd_callout) can change directory, causing
stubs to be unable to locate EVENTSCRIPTS_TESTS_VAR_DIR if it is
relative.

Don't force TEST_VAR_DIR to be absolute because this can result in
Unix domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
63a4c634 by Martin Schwenke at 2018-12-18T01:02:04Z
ctdb-tests: Force symbolic link targets to be absolute

If CTDB_BASE is relative then the symbolic link target will be
incorrect.

Don't force CTDB_BASE to be absolute because this can result in Unix
domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
108aca0a by Martin Schwenke at 2018-12-18T01:02:04Z
ctdb-event: Declare and construct data_script only if needed

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
61b54193 by Martin Schwenke at 2018-12-18T01:02:04Z
ctdb-event: Force symbolic link targets to be absolute

If CTDB_BASE is relative then symbolic link targets will be incorrect.

Don't force CTDB_BASE to be absolute because this can result in Unix
domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
1ed91f0e by Martin Schwenke at 2018-12-18T04:31:00Z
ctdb-tests: Do not force TEST_VAR_DIR to be absolute

This can result in Unix domain socket names that are too long.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Tue Dec 18 05:31:00 CET 2018 on sn-devel-144

- - - - -
1e061ff1 by Amitay Isaacs at 2018-12-18T06:12:09Z
ctdb-tool: Avoid data uninitialized warnings

../../tools/ctdb.c: In function 'str_to_data':
../../tools/ctdb.c:624: warning: 'data.dsize' may be used uninitialized in this function
../../tools/ctdb.c:624: warning: 'data.dptr' may be used uninitialized in this function

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
9912709e by Amitay Isaacs at 2018-12-18T06:12:09Z
ctdb-build: Use open() instead of file() for python3

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
4443124f by Amitay Isaacs at 2018-12-18T06:12:09Z
ctdb-packaging: Call waf with python wrapper

This allows to build packages even when python3 is not available by
setting PYTHON variable.

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
59e244c9 by Amitay Isaacs at 2018-12-18T06:12:09Z
ctdb-packaging: Match configure command as per spec file

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
9bdd6814 by Amitay Isaacs at 2018-12-18T06:12:09Z
ctdb-packaging: Update library versions to upstream versions

Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>

- - - - -
f1b594dc by Martin Schwenke at 2018-12-18T06:12:09Z
ctdb-daemon: Do not force full vacuum on first vacuuming run

When the number of fast path vacuuming runs is 0 then a full vacuuming
run is done.  This means the first one is a full run, which is almost
certainly not what is intended.

Combine the 2 conditionals to only flag a full vacuuming run when the
count exceeds the configured limit.  This means that the
full_vacuum_run flag is set in both parent and child, but this is
harmless... and is better than getting it wrong.

Also tweak the comparison to be less-than-or-equal, since the zeroth
run needs to be counted.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
2e3ad8c2 by Martin Schwenke at 2018-12-18T06:12:10Z
ctdb-tests: Minimise chances of test interfering with itself

Checking that the database contains 0 records cause a traverse.  This
may take a lock and cause vacuuming to fail (or be deferred for a
particular record/chain).  Minimise the chance of this happening by
only checking for 0 records every 10 seconds.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
cdca0d7e by Martin Schwenke at 2018-12-18T06:12:10Z
ctdb-daemon Add extra debug during record deletion for vacuuming

It isn't currently possible to distinguish these 2 cases.

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

- - - - -
944c92a1 by Martin Schwenke at 2018-12-18T09:13:50Z
ctdb-daemon: Modernise debug during record deletion for vacuuming

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Tue Dec 18 10:13:50 CET 2018 on sn-devel-144

- - - - -
90fab07f by Günther Deschner at 2018-12-18T19:11:07Z
s3-smbd: avoid assuming fsp is always intact after close_file call.

Instead use the already copied smb_fname directly.

https://bugzilla.samba.org/show_bug.cgi?id=13720

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>

Autobuild-User(master): Günther Deschner <gd at samba.org>
Autobuild-Date(master): Tue Dec 18 20:11:07 CET 2018 on sn-devel-144

- - - - -
c817deca by Swen Schillig at 2018-12-19T03:51:26Z
s4: Remove double init of kerberos error table

The initialization of the kerberos error table
is already performed in smb_krb5_init_context_basic(),
therefore, it can be removed from of its callees.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Dec 19 04:51:27 CET 2018 on sn-devel-144

- - - - -
8741af7a by Gary Lockyer at 2018-12-19T03:52:59Z
s4 messaging tests: Add inject fault command

Test for processing of the smbcontrol inject fault message in the samba
daemon.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8167486b by Gary Lockyer at 2018-12-19T03:52:59Z
s4 messaging: support smbcontrol inject fault command

Add support of the smbcontrol inject fault command to the samba daemon.
This is useful for manual testing of process restart etc.

command is only enabled for developer and self test builds

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b578fad8 by Gary Lockyer at 2018-12-19T03:52:59Z
s4 messaging tests: Tests for smbcontrol sleep command

Add a sleep command that pauses the target process for the specified
number seconds

This command is only enabled on developer and self test builds.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ff9052c9 by Gary Lockyer at 2018-12-19T03:52:59Z
s3 smbcontrol: Add sleep command

Add a sleep command that pauses the target process for the specified
number of seconds

This command is only enabled on developer and self test builds.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a70deab0 by Gary Lockyer at 2018-12-19T03:52:59Z
s3 server: Add support for smbcontrol sleep

Add a sleep command that pauses the target process for the specified number
of seconds

This command is only enabled on self test and developer builds.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b3e4e094 by Gary Lockyer at 2018-12-19T03:52:59Z
s4 messaging: Add support for smbcontrol sleep

Add a sleep command that pauses the target process for the specified
number of seconds

This command is only enabled on developer and self test builds.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
51189858 by Martin Schwenke at 2018-12-19T07:08:28Z
lib/util: Count a trailing line that doesn't end in a newline

If the final line of a file does not contain a newline then it isn't
included in the line count.

Change i to point to the next slot in the array instead of the current
one.  This means that that the current line won't be thrown away if no
newline is seen.

Without changing i to unsigned int, the -O3 --picky -developer build
fails with:

[ 745/4136] Compiling lib/util/util_file.c

==> /builds/samba-team/devel/samba/samba-o3.stderr <==
../../lib/util/util_file.c: In function ‘file_lines_parse’:
../../lib/util/util_file.c:251:8: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow]
  while (i > 0 && ret[i-1][0] == 0) {
        ^
cc1: all warnings being treated as errors

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13717

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed Dec 19 08:08:28 CET 2018 on sn-devel-144

- - - - -
b1ad5a88 by Andreas Schneider at 2018-12-19T11:40:40Z
krb5_wrap: Fix bit shifting

lib/krb5_wrap/enctype_convert.c:93:27: runtime error: left shift of 1 by
31 places cannot be represented in type 'int'

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Gary Lockyer <gary at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Wed Dec 19 12:40:40 CET 2018 on sn-devel-144

- - - - -
d0b5e216 by Samuel Cabrero at 2018-12-19T11:42:09Z
selftest: Create included files during provision

Files included from smb.conf have to exists, otherwise python fails to
load the configuration. Found while trying to run a python test before
samba3.blackbox.smbd_error creates the included file.

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e8144129 by Samuel Cabrero at 2018-12-19T11:42:09Z
selftest: Add a new base class for ntlm_auth tests

The class is based on test_ntlm_auth.py script.

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Pair-programmed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
247592f7 by Samuel Cabrero at 2018-12-19T11:42:09Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth and ntlm_auth with specified domain

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4a11ab37 by Samuel Cabrero at 2018-12-19T11:42:09Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth against winbindd

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d3fd3d01 by Samuel Cabrero at 2018-12-19T11:42:10Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth with NTLMSSP client and gss-spnego server

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c9a5bf3c by Samuel Cabrero at 2018-12-19T11:42:10Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth with NTLMSSP gss-spnego-client and gss-spnego server

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4ce9371f by Samuel Cabrero at 2018-12-19T11:42:10Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth with NTLMSSP gss-spnego-client and gss-spnego server against winbind

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
06d101e3 by Samuel Cabrero at 2018-12-19T11:42:11Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth ccached credentials with NTLMSSP client and gss-spnego server

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
1f704496 by Samuel Cabrero at 2018-12-19T11:42:11Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth against winbindd with require-membership-of

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Signed-off-by: Noel Power <noel.power at suse.com>

- - - - -
8d11a54e by Samuel Cabrero at 2018-12-19T11:42:11Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth with NTLMSSP gss-spnego-client and gss-spnego server against winbind with require-membership-of

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
377e87a2 by Samuel Cabrero at 2018-12-19T11:42:12Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth against winbindd with failed require-membership-of

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
caa50530 by Samuel Cabrero at 2018-12-19T11:42:12Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth with NTLMSSP gss-spnego-client and gss-spnego server against winbind with failed require-membership-of

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
dfa14927 by Samuel Cabrero at 2018-12-19T11:42:12Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth plaintext authentication with require-membership-of

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3794c1c5 by Samuel Cabrero at 2018-12-19T11:42:12Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth plaintext authentication with failed require-membership-of

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
28ea2f72 by Samuel Cabrero at 2018-12-19T11:42:13Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth ntlm-server-1 with fixed password

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7c0c683b by Samuel Cabrero at 2018-12-19T11:42:13Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth ntlm-server-1 with incorrect fixed password

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8d4258f4 by Samuel Cabrero at 2018-12-19T11:42:13Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth ntlm-server-1 with plaintext password against winbind

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3ae7095b by Samuel Cabrero at 2018-12-19T11:42:13Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth ntlm-server-1 with plaintext password against winbind but wrong sid

Port ntlm_auth bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
91c4f260 by Samuel Cabrero at 2018-12-19T11:42:14Z
tests/ntlm_auth: Port ntlm_auth tests to python: ntlm_auth ntlm-server-1 with incorrect fixed password against winbind

Port ntlm_auth bash script tests to python and remove bash test script

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
129ab34b by Samuel Cabrero at 2018-12-19T11:42:14Z
tests/ntlm_auth: Port ntlm_auth_krb5 tests to python

Port ntlm_auth_krb5 bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4846d4a7 by Samuel Cabrero at 2018-12-19T11:42:14Z
selftest: Remove test_ntlm_auth.py helper

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
fe7ab7d5 by Samuel Cabrero at 2018-12-19T15:21:32Z
tests/ntlm_auth: Port ntlm_auth_diagnostics tests to python

Port ntlm_auth_diagnostics bash script tests to python

Signed-off-by: Samuel Cabrero <scabrero at suse.de>
Reviewed-by: Noel Power <npower at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Wed Dec 19 16:21:32 CET 2018 on sn-devel-144

- - - - -
49dc04f9 by Björn Jacke at 2018-12-19T19:58:52Z
samba-tool: don't print backtrace on simple DNS errors

samba-tool throws backtraces even for simple DNS error
messages, we should not frighten users for no good reason.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13721

Signed-off-by: Bjoern Jacke <bj at sernet.de>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Björn Jacke <bj at sernet.de>
Autobuild-Date(master): Wed Dec 19 20:58:52 CET 2018 on sn-devel-144

- - - - -
a800baec by Swen Schillig at 2018-12-19T20:49:29Z
Add MIT kerberos tracing capability

HEIMDAL kerberos offers already tracing via a logging facility
through smb_krb5_init_context().
MIT kerberos offers to register a callback via krb5_set_trace_callback
with which tracing information can be routed to a common logging facility.
This is now integrated into smb_krb5_init_context_basic() offering
the same functionality for both kerberos fragrances.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
5056b962 by Swen Schillig at 2018-12-19T20:49:29Z
lib: Add kerberos tracing

Add krb5 tracing to samba krb5 wrapper.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
2cc561fb by Swen Schillig at 2018-12-19T20:49:29Z
client: Add kerberos tracing

Replace kerberos context initialization from
raw krb5_init_context() to smb_krb5_init_context_basic()
which is adding common tracing as well.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
3df7789e by Swen Schillig at 2018-12-19T20:49:29Z
libads: Add kerberos tracing

Replace kerberos context initialization from
raw krb5_init_context() to smb_krb5_init_context_basic()
which is adding common tracing as well.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
8887a680 by Swen Schillig at 2018-12-19T20:49:29Z
libnet: Add kerberos tracing

Replace kerberos context initialization from
raw krb5_init_context() to smb_krb5_init_context_basic()
which is adding common tracing as well.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
9f338647 by Swen Schillig at 2018-12-19T20:49:29Z
librpc: Add kerberos tracing

Replace kerberos context initialization from
raw krb5_init_context() to smb_krb5_init_context_basic()
which is adding common tracing as well.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
bf675938 by Swen Schillig at 2018-12-19T20:49:29Z
passdb: Add kerberos tracing

Replace kerberos context initialization from
raw krb5_init_context() to smb_krb5_init_context_basic()
which is adding common tracing as well.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
5eefb088 by Swen Schillig at 2018-12-19T20:49:29Z
utils: Add kerberos tracing

Replace kerberos context initialization from
raw krb5_init_context() to smb_krb5_init_context_basic()
which is adding common tracing as well.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
0b0a22dd by Swen Schillig at 2018-12-19T20:49:29Z
winbindd: Add kerberos tracing

Replace kerberos context initialization from
raw krb5_init_context() to smb_krb5_init_context_basic()
which is adding common tracing as well.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

- - - - -
357fe04b by Swen Schillig at 2018-12-20T00:31:17Z
s4: Add kerberos tracing

Replace kerberos context initialization from
raw krb5_init_context() to smb_krb5_init_context_basic()
which is adding common tracing as well.

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>

Autobuild-User(master): Christof Schmitt <cs at samba.org>
Autobuild-Date(master): Thu Dec 20 01:31:17 CET 2018 on sn-devel-144

- - - - -
7a2562fd by Ralph Boehme at 2018-12-20T02:19:25Z
debug: move some definitions around

They will be needed by some function in a subsequent commit.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3b293c66 by Ralph Boehme at 2018-12-20T02:19:25Z
debug: rename DEBUGLEVEL_CLASS variable to dbgc_config

Variable names should not be given in uppercase.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d346cf94 by Ralph Boehme at 2018-12-20T02:19:25Z
debug: README.Coding fixes

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
63f33076 by Ralph Boehme at 2018-12-20T02:19:26Z
debug: convert dbgc_config to an array of struct debug_class

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e6655f08 by Ralph Boehme at 2018-12-20T02:19:26Z
debug: track current debug message class

This is analog to current_msg_level.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
240b08c4 by Ralph Boehme at 2018-12-20T02:19:26Z
debug: factor out a function that opens and closes the new and old logfile

The new function reopen_one_log() will also be used for per-class
logfiles in subsequent commmits.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2bc7e254 by Ralph Boehme at 2018-12-20T02:19:26Z
debug: add an empty line

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4341f240 by Ralph Boehme at 2018-12-20T02:19:26Z
debug: add a call to debug_parse_levels() to reopen_logs()

This allows correct refresh of the "log level" setting when reloading config.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
73696747 by Ralph Boehme at 2018-12-20T02:19:26Z
debug: factor out logfile size check

The new function will also be used for upcoming per-debug-class logfiles.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
43c69351 by Ralph Boehme at 2018-12-20T02:19:26Z
debug: add logfile and fd to struct debug_class

Initialized to -1. Already checked in debug_file_log() without affecting
behaviour until subsequent commits set per-debug-class fds.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
abfb3c6b by Ralph Boehme at 2018-12-20T02:19:27Z
debug: add resource cleanup for per debug-class logfiles

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
249bdd93 by Ralph Boehme at 2018-12-20T02:19:27Z
debug: add support for per debug-class logfiles

This adds support for per debug-class logfiles to the function parsing
the "log level" option.

The enhanced syntax is:

  log level = CLASS:LEVEL[@PATH] [CLASS:LEVEL[@PATH] ... ]

Eg

  log level = full_audit:1@/var/log/audit.logfile

While the option is already parsed and stored in in the dbgc_config[]
array, the feature is still effectively disabled, as
reopen_logs_internal() still doesn't open the per debug-class logfiles.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d792f15f by Ralph Boehme at 2018-12-20T02:19:27Z
debug: update need_to_check_log_size() for per debug-class logfiles

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
781f5a95 by Ralph Boehme at 2018-12-20T02:19:27Z
debug: update logsize checking for per debug-class logfiles

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3dbda632 by Ralph Boehme at 2018-12-20T02:19:27Z
debug: remove fd and debugf from state, use dbgc_config[DBGC_ALL]

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
08d37b4a by Ralph Boehme at 2018-12-20T02:19:27Z
debug: enable per debug-class logfiles

This finally enables per debug-class logfiles by hooking into
reopen_logs_internal() calls to reopen_one_log() per configured
debug-class.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
35349fec by Ralph Boehme at 2018-12-20T02:19:27Z
docs-xml: document "log level" changes

"log level" now takes an optional per debug-class logfile:

  log level = 1 full_audit:1@/var/log/audit.log winbind:2

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5c928d7c by Philipp Gesang at 2018-12-20T02:19:28Z
lib/audit_logging: actually create talloc

Heal damage of 79f494e51e..

That context is being passed around and freed but is never
actually allocated on that stack.

Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1b263ed6 by Günther Deschner at 2018-12-20T06:18:20Z
s3-vfs-streams_xattr: add close call

https://bugzilla.samba.org/show_bug.cgi?id=13725

We cannot always rely on vfs_default to close the fake fds. This mostly is
relevant when used with another non-local VFS filesystem module such as
gluster.

Guenther

Signed-off-by: Günther Deschner <gd at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Dec 20 07:18:20 CET 2018 on sn-devel-144

- - - - -
865538fa by Stefan Metzmacher at 2018-12-20T07:41:25Z
s3:auth: ignore create_builtin_guests() failing without a valid idmap configuration

This happens on standalone servers, where winbindd is automatically
started by init scripts if it's installed. But it's not really
used and may not have a valid idmap configuration (
"idmap config * : range" has no default!)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13697

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
f3bac8c9 by Stefan Metzmacher at 2018-12-20T07:41:25Z
s3:auth_winbind: remove fallback to optional backend

This is not possible anymore, as the trustdomain backend
was removed in commit 75c152c0d764165a4a9dd0a85390af063dd0192a.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13722
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13723

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
ec3adc1e by Stefan Metzmacher at 2018-12-20T07:41:25Z
s3:auth_winbind: return NT_STATUS_NO_LOGON_SERVERS if winbindd is not available

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13722
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13723

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

- - - - -
63dc6076 by Stefan Metzmacher at 2018-12-20T11:15:09Z
s3:auth_winbind: ignore a missing winbindd as NT4 PDC/BDC without trusts

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13722

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>

Autobuild-User(master): Alexander Bokovoy <ab at samba.org>
Autobuild-Date(master): Thu Dec 20 12:15:09 CET 2018 on sn-devel-144

- - - - -
7a35a687 by Andreas Schneider at 2018-12-20T11:16:39Z
waf: Add missing libreplace deps to pyldb*

This will require memset_s() because of a later commit moving
ZERO_STRUCT to use memset_s().

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7eaa5968 by Andreas Schneider at 2018-12-20T11:16:39Z
wafsamba: Do not remove BUILTINS as duplicates

BUILTINS add object files to the target, so we can't remove them as
duplicates.

The issue e.g happens when tevent wants to link libreplace:

   20:04:10 deps removing dups from tevent of type LIBRARY: {'replace'}
   also in LIBRARY talloc

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Alexander Bokovoy <ab at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9968217b by Andreas Schneider at 2018-12-20T11:16:39Z
replace: Add memset_s to replacement functions

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
255117a1 by Andreas Schneider at 2018-12-20T11:16:39Z
replace: Cleanup comments for ZERO_*

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3611f18f by Andreas Schneider at 2018-12-20T11:16:39Z
replace: Use memset_s for ZERO_* macros

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e99ca99d by Andreas Schneider at 2018-12-20T11:16:40Z
lib:util: Cleanup comments in memory.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
161d14f2 by Andreas Schneider at 2018-12-20T11:16:40Z
lib:util: Sync memory.h with replace.h

We can't remove memory.h as this is a public header file. So we need to
duplicate them from replace.h

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6b2c6c0e by Andreas Schneider at 2018-12-20T11:16:40Z
libcli:auth: Use C99 initializers or ZERO_ARRAY instead of ZERO_STRUCT

ZERO_STRUCT is not wrong here, it will give the same result, but better
use macros with correct naming as it makes clear what happens when you
read the code.

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
39bff1f9 by Andreas Schneider at 2018-12-20T11:16:40Z
libcli:smb: Avoid explicit ZERO_STRUCT

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f9c15350 by Andreas Schneider at 2018-12-20T15:11:02Z
lib:util: Use memset_s() in data_blob_clear()

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Dec 20 16:11:03 CET 2018 on sn-devel-144

- - - - -
454d7d00 by Volker Lendecke at 2018-12-20T22:40:23Z
net: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
69a3b899 by Volker Lendecke at 2018-12-20T22:40:24Z
smbcacls: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
444fc0bf by Volker Lendecke at 2018-12-20T22:40:24Z
smbd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
1d5c00a3 by Volker Lendecke at 2018-12-20T22:40:24Z
libads: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
534620d1 by Volker Lendecke at 2018-12-20T22:40:24Z
winbind: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
09efea3c by Volker Lendecke at 2018-12-20T22:40:24Z
groupdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
64388196 by Volker Lendecke at 2018-12-20T22:40:24Z
passdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
998d1d1a by Volker Lendecke at 2018-12-20T22:40:24Z
pdb_smbpasswd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4cd68f3d by Volker Lendecke at 2018-12-20T22:40:24Z
pdb_ldap: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
c45c6d09 by Volker Lendecke at 2018-12-20T22:40:24Z
lookup_sid: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
59f29acb by Volker Lendecke at 2018-12-20T22:40:25Z
vfs: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
6af7d7ff by Volker Lendecke at 2018-12-20T22:40:25Z
auth3: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
da3e3c5e by Volker Lendecke at 2018-12-20T22:40:25Z
lib: Avoid sid_string_dbg

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
4d4a59ec by Volker Lendecke at 2018-12-20T22:40:25Z
lib: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
1ab6c336 by Volker Lendecke at 2018-12-20T22:40:25Z
libnet: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
d425cd5b by Volker Lendecke at 2018-12-20T22:40:25Z
wkssvc: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
58f76ab1 by Volker Lendecke at 2018-12-20T22:40:25Z
winbindd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
31da287b by Volker Lendecke at 2018-12-20T22:40:25Z
spoolss: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
fa6690e9 by Volker Lendecke at 2018-12-20T22:40:25Z
lsasrv: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
fc51f824 by Volker Lendecke at 2018-12-20T22:40:26Z
samr: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
131ee1b2 by Volker Lendecke at 2018-12-20T22:40:26Z
lib: Remove sid_string_dbg

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
dd6175a2 by Aaron Haslett at 2018-12-20T22:40:26Z
dns: test for treating fully qualified zones same as unqualified

Failing test that checks if fully qualified zone names are treated the same
as unqualified zone names by the dns zone creation RPC method. Fix to follow.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13214
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ebc42cb1 by Aaron Haslett at 2018-12-20T22:40:26Z
dns: treating fully qualified and unqualified zone as identical.

"zone.com." and "zone.com" should be treated as the same zone.  This patch
picks the unqualified representation as standard and enforces it, in order to
match BIND9 behaviour.
Note: This fixes the failing test added previously, but that test still fails
on the rodc test target so we modify the expected failure but don't remove it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13442
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bd2d6b2b by Gary Lockyer at 2018-12-20T22:40:26Z
WHATSNEW: JSON logging changes

Add details of the Windows Event Id's and Logon Types added to the
Authorization qnd PasswordChange JSON messages.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
05c981f0 by Gary Lockyer at 2018-12-20T22:40:26Z
tests (audit_auth)_log: PEP8 cleanup.

Remove Flake8 warnings from the audit and authentication JSON log
tests.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
39087160 by Gary Lockyer at 2018-12-21T02:39:24Z
s4 messaging tests: Fix race condition in smbcontrol tests

The test for the smbcontrol sleep command and the inject fault command
both used the "rpc_server" process as a target.  As the inject fault
command caused the process to restart there was a race condition between
the process restarting and the sleep test running.  To prevent this the
tests now use different target processes.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Fri Dec 21 03:39:24 CET 2018 on sn-devel-144

- - - - -
ba016939 by Günther Deschner at 2018-12-21T06:20:48Z
s3-vfs-fruit: add close call

https://bugzilla.samba.org/show_bug.cgi?id=13725

We cannot always rely on vfs_default to close the fake fds. This mostly is
relevant when used with another non-local VFS filesystem module such as
gluster.

Guenther

Signed-off-by: Günther Deschner <gd at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Dec 21 07:20:49 CET 2018 on sn-devel-144

- - - - -
96c03c75 by Aaron Haslett at 2018-12-21T06:21:19Z
vlv: tests for delete, add, and modify cases

More vlv testing for cases involving modifying, deleting, and adding records
while observing the effect on already initialised views.

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
97580700 by Aaron Haslett at 2018-12-21T06:21:19Z
paged results: new paged results module using GUID list

Replacing paged results module to use GUID list instead of storing
result list in memory, in order to improve memory performance.

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
22f1c400 by Aaron Haslett at 2018-12-21T10:10:30Z
paged results: testing suite for new paged results module

Testing the new GUID list based paged results module

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Gary Lockyer <gary at samba.org>
Autobuild-Date(master): Fri Dec 21 11:10:30 CET 2018 on sn-devel-144

- - - - -
b99b5140 by Andrew Bartlett at 2018-12-21T21:16:09Z
build: Remove --timestamp-dependencies (BROKEN)

Remove this code marked as broken, we do not need broken configure options making Samba
appear to be more complex than it already is.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
87a8325a by Gary Lockyer at 2018-12-21T21:16:09Z
s4 group_audit: Add Windows Event Id's to Group membership changes

Generate a GroupChange event when a user is created with a PrimaryGroup
membership.  Log the windows event id in the JSON GroupChange message.

Event Id's supported are:
	4728	A member was added to a security enabled global group
	4729	A member was removed from a security enabled global
		group
	4732	A member was added to a security enabled local group
	4733	A member was removed from a security enabled local group
	4746	A member was added to a security disabled local group
	4747	A member was removed from a security disabled local group
	4751	A member was added to a security disabled global group
	4752	A member was removed from a security disabled global
		group
	4756	A member was added to a security enabled universal
		group
	4757	A member was removed from a security enabled universal
		group
	4761	A member was added to a security disabled universal
		group
	4762	A member was removed from a security disabled universal
		group

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
27273a55 by Gary Lockyer at 2018-12-21T21:16:09Z
tests group_audit: PEP8 cleanup.

Remove Flake8 warnings from the group  audit JSON log tests.

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
42dde0bd by Andrew Bartlett at 2018-12-22T00:58:48Z
group_audit: Ensure we still log membership changes (with an error) where status != LDB_SUCCESS

This restores the previous behaviour.  It causes (only) the event ID
to be omitted if status != LDB_SUCCESS or there was a problem getting
the group type.

Errors at this stage are exceedingly rare, because the values have
already been checked by the repl_meta_data module, but this is
cosistent with the rest of the module again.

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sat Dec 22 01:58:48 CET 2018 on sn-devel-144

- - - - -
b9496ddb by Björn Jacke at 2018-12-22T02:11:13Z
winbind_nss_aix: support also S_GROUPSIDS

which is used by lsuser up to AIX 5.2, see also https://bugzilla.samba.org/show_bug.cgi?id=5157

Signed-off-by: Bjoern Jacke <bj at sernet.de>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
a9b71194 by Stefan Metzmacher at 2018-12-22T02:11:14Z
nsswitch/winbind_nss_aix: reimplement fetching the SID of a user

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Bjoern Jacke <bj at sernet.de>

- - - - -
245b494c by Bjoern Jacke at 2018-12-22T02:11:14Z
winbind introduce WBFLAG_FROM_NSS

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Bjoern Jacke <bj at sernet.de>

- - - - -
f95495e1 by Stefan Metzmacher at 2018-12-22T02:11:14Z
winbind: honor WBFLAG_FROM_NSS along with winbind enum users/groups

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Bjoern Jacke <bj at sernet.de>

- - - - -
4a736850 by Bjoern Jacke at 2018-12-22T02:11:14Z
nsswitch/winbind_nss_linux.c use WBFLAG_FROM_NSS

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Bjoern Jacke <bj at sernet.de>

- - - - -
fcda61bb by Björn Jacke at 2018-12-22T02:11:14Z
winbind_nss_netbsd: use WBFLAG_FROM_NSS

Signed-off-by: Bjoern Jacke <bj at sernet.de>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
faf50b2d by Björn Jacke at 2018-12-22T02:11:14Z
winbind_nss_solaris: use WBFLAG_FROM_NSS

Signed-off-by: Bjoern Jacke <bj at sernet.de>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
c36cf69d by Björn Jacke at 2018-12-22T02:11:14Z
winbind_nss_aix: use WBFLAG_FROM_NSS

Signed-off-by: Bjoern Jacke <bj at sernet.de>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
2e1bc87b by Björn Jacke at 2018-12-22T02:11:14Z
winbind_nss_aix: add incomplete attr_flag initializations

Found by Jürgen Starek, see https://bugzilla.samba.org/show_bug.cgi?id=5157

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Bjoern Jacke <bj at sernet.de>

- - - - -
ccd4759a by Bjoern Jacke at 2018-12-22T05:22:22Z
nss/waf: check for secmethod_table.method_{attrlist,version} for aix >= 5.2

The original commit 02c9b46fab46ab401a3cf6bb74c8260801c41032 was not ported to waf yet.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Bjoern Jacke <bj at sernet.de>

Autobuild-User(master): Björn Jacke <bj at sernet.de>
Autobuild-Date(master): Sat Dec 22 06:22:22 CET 2018 on sn-devel-144

- - - - -
48815cc1 by Stefan Metzmacher at 2018-12-23T17:15:19Z
s3:libads: do an early return if we don't have a password for ads_kinit_password()

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4114d2d0 by Stefan Metzmacher at 2018-12-23T17:15:19Z
s4:librpc: add python witness bindings

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8b153f84 by Stefan Metzmacher at 2018-12-23T17:15:19Z
s4:librpc: add python spoolss bindings

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e27b6276 by Stefan Metzmacher at 2018-12-23T17:15:19Z
s4:librpc: add python winspool bindings

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
deb0c2bd by Stefan Metzmacher at 2018-12-23T17:15:19Z
s3:pylibsmb: add settimeout()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c91446cf by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/tests: rename dcerpc/string.py -> string_tests.py

Otherwise it's not possible to run the raw_protocol tests anymore:

python/samba/tests/dcerpc/raw_protocol.py
Traceback (most recent call last):
  File "python/samba/tests/dcerpc/raw_protocol.py", line 26, in <module>
    import samba.dcerpc.dcerpc as dcerpc
  File "bin/python/samba/__init__.py", line 32, in <module>
    from samba.compat import string_types
  File "bin/python/samba/compat.py", line 151, in <module>
    from urllib import quote as urllib_quote
  File "/usr/lib/python2.7/urllib.py", line 25, in <module>
    import string
  File "/abs/path/samba/python/samba/tests/dcerpc/string.py", line 22, in <module>
    # Some strings for ctype-style character classification
  File "bin/python/samba/tests/__init__.py", line 36, in <module>
    from samba.compat import text_type
ImportError: cannot import name text_type

This allows the following again:

SMB_CONF_PATH=/dev/null \
    SERVER=172.31.9.188 \
    TARGET_HOSTNAME=w2012r2-188.w2012r2-l6.base \
    USERNAME=administrator \
    PASSWORD=A1b2C3d4 \
    DOMAIN=W2012R2-L6 \
    REALM=W2012R2-L6.BASE \
    python/samba/tests/dcerpc/raw_protocol.py -v -f TestDCERPC_BIND

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
71450d85 by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/raw_protocol: rename test_spnego_packet_bind_sign_privacy => test_spnego_packet_bind_seal

This makes it consistent with other tests like
test_spnego_integrity_bind_seal.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
cc2dc2e2 by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/raw_testcase: disconnect on tearDown() of RawDCERPCTest

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fc4a8c17 by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/raw_protocol: explicitly disconnect additional connections

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
460a9d37 by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/raw_testcase: add assertPadding() that allows IGNORE_RANDOM_PAD=1

Sometimes Windows returns non zero bytes in padding fields,
we won't allow that by default, but IGNORE_RANDOM_PAD=1 will
will only do the length check.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dae9c53f by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/raw_testcase: make use of assertPadding()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
da82aa2f by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/raw_protocol: make use of assertPadding()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1ba90dae by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/raw_testcase: let do_single_request() check stub length against alloc_hint

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
59b84d64 by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/raw_protocol: let self._test_spnego_bind_auth_level() return auth_context

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
20ed4bd9 by Stefan Metzmacher at 2018-12-23T17:15:20Z
py:dcerpc/raw_testcase: pass auth_context and stub_len to parse_auth() in order to assert

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8fc1f562 by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_protocol: rename _test_spnego_bind_auth_level to_test_auth_bind_auth_level

We now pass down dcerpc.DCERPC_AUTH_TYPE_SPNEGO from callers
instead of having SPNEGO specific functions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6b1c3858 by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_protocol: rename _test_spnego_signing_auth_level_request to _test_auth_signing_auth_level_request

We now pass down dcerpc.DCERPC_AUTH_TYPE_SPNEGO from callers
instead of having SPNEGO specific functions.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fed1cdbe by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_testcase: add check_response_auth() helper function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
168aa7c3 by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_testcase: add generate_request_auth() helper function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0b741c49 by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_testcase: use check_response_auth() in do_single_request()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
87197f0c by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_testcase: use generate_request_auth() in do_single_request()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3924924e by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_testcase: use require DOMAIN/REALM in get_user_creds()

This is the usage now:

SMB_CONF_PATH=/dev/null \
    SERVER=172.31.9.188 \
    TARGET_HOSTNAME=w2012r2-188.w2012r2-l6.base \
    USERNAME=administrator \
    PASSWORD=A1b2C3d4 \
    DOMAIN=W2012R2-L6 \
    REALM=W2012R2-L6.BASE \
    IGNORE_RANDOM_PAD=1 \
    python/samba/tests/dcerpc/raw_protocol.py -v -f TestDCERPC_BIND

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
68e5dbe7 by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_testcase: prepare do_generic_bind() for raw NTLMSSP and Kerberos authentication

They just use 3 legs (messages) for the authentication, while SPNEGO
uses 2 or 4 messages.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
78a5e4ec by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_testcase: prepare get_auth_context_creds() and do_generic_bind() for header signing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f819df4d by Stefan Metzmacher at 2018-12-23T17:15:21Z
py:dcerpc/raw_protocol: test signing with and without header signing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6574d2da by Stefan Metzmacher at 2018-12-23T17:15:22Z
py:dcerpc/raw_protocol: test signing also with raw NTLMSSP and Kerberos

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7a83cb02 by Stefan Metzmacher at 2018-12-23T17:15:22Z
py:dcerpc/raw_testcase: add start_with_alter to do_generic_bind()

This will allow do_generic_bind() to be used to test
security context multiplexing.

Check with git show -w

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7fb5c575 by Stefan Metzmacher at 2018-12-23T17:15:22Z
py:dcerpc/raw_testcase: support DCERPC_AUTH_LEVEL_CONNECT in do_single_request()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
196cc4db by Stefan Metzmacher at 2018-12-23T17:15:22Z
py:dcerpc/raw_testcase: maintain self.max_{xmit,recv}_frag

This was it's easier to alter once add support for SMB connections.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
df7d4787 by Stefan Metzmacher at 2018-12-23T17:15:22Z
py:dcerpc/raw_testcase: maintain self.secondary_address

This was it's easier to alter once add support for SMB connections.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b34eb437 by Stefan Metzmacher at 2018-12-23T17:15:22Z
py:dcerpc/raw_testcase: add helper functions for ncacn_np: SMB connection support

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
be2a6731 by Stefan Metzmacher at 2018-12-23T20:33:51Z
auth/gensec: enforce that all DCERPC contexts support SIGN_PKT_HEADER

That's currently always the case and will simplifies the callers.

WORKS now???
TDB_NO_FSYNC=1 buildnice make -j test FAIL_IMMEDIATELY=1 SOCKET_WRAPPER_KEEP_PCAP=1 TESTS='samba4.rpc.lsa.secrets.*ncacn_np.*Kerberos.*Samba3.*fl2000dc'
and
TDB_NO_FSYNC=1 buildnice make -j test FAIL_IMMEDIATELY=1 SOCKET_WRAPPER_KEEP_PCAP=1 TESTS='samba3.rpc.lsa.*ncacn_ip_tcp.*nt4_dc'

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sun Dec 23 21:33:51 CET 2018 on sn-devel-144

- - - - -
3d016d6f by Stefan Metzmacher at 2019-01-01T15:02:05Z
Happy New Year 2019!

Signed-off-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Tue Jan  1 16:02:05 CET 2019 on sn-devel-144

- - - - -
f353c930 by Bjoern Jacke at 2019-01-02T13:10:41Z
waf: let CHECK_SIZEOF check for 64 bit also

wafsamba: utmp can be 64 bit also (like on AIX)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12017

Signed-off-by: Bjoern Jacke <bj at sernet.de>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Wed Jan  2 14:10:41 CET 2019 on sn-devel-144

- - - - -
b7d350b4 by Bjoern Jacke at 2019-01-04T15:29:50Z
s3:smbd: Fix build on AIX

AIX makes a define of ip_len in netinet/ip.h (sic!)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13731

Signed-off-by: Bjoern Jacke <bj at sernet.de>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Fri Jan  4 16:29:50 CET 2019 on sn-devel-144

- - - - -
6fff2c26 by Tim Beale at 2019-01-07T00:23:07Z
s3:pylibsmb: Make lp a mandatory param for the SMB connection

Currently establishing the SMB connection relies on having initialized
the global source3 loadparm.

This patch makes the lp param mandatory, so that you always have to pass
the parameter in when establishing the SMB connection.

It also makes the source3 API more consistent with the current source4
API, which will make it easier to replace the source4 version later.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8096419c by Tim Beale at 2019-01-07T00:23:07Z
s3:pylibsmb: Add .unlink() API to SMB Py bindings

Add a basic .unlink() API to the source3 bindings. This is based on the
source4 python bindings, but uses the source3 client library APIs.
(We use a helper function to do most of the work, because we will need
to reuse it in order to support the deltree API).

Update the source4 test to use the source3 API. We will gradually
convert it over, and then delete the source4 python bindings.

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6a40e538 by Tim Beale at 2019-01-07T00:23:07Z
s3:pylibsmb: Add .mkdir(), .rmdir() APIS to SMB py bindings

I kept these separate from the chkpath API because it's a nice way to
use the old s4 API in the tests to verify the new s3 API works
correctly.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c4dee343 by Tim Beale at 2019-01-07T00:23:07Z
s3:pylibsmb: Add .chkpath() API to SMB py bindings

Note that this is checking the existence of *directories*, not *files*.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
40f4e527 by Tim Beale at 2019-01-07T00:23:07Z
s3:pylibsmb: Consolidate .readdir()/.list() py bindings API

Rename the .readdir API to .list and make the parameters match up with
the s4 python bindings, i.e. 'mask' is optional and separate from the
directory parameter.

Note that nothing uses the .readdir source3 API currently, so it's safe
to rename.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
2033208a by Tim Beale at 2019-01-07T00:23:07Z
s3:pylibsmb: Split out code into list_helper()

Refactor out the code so we can re-use it for the synchronous API as
well.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
05716103 by Tim Beale at 2019-01-07T00:23:07Z
s3:pylibsmb: Split code out into do_listing() helper

We'll want to reuse the directory listing function for the deltree API.
This patch splits the bulk of the work out into a do_listing() helper
function.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3ac4866d by Tim Beale at 2019-01-07T00:23:07Z
s3:pylibsmb: Make .list() work for SMBv2

In order for .list() to work on an SMBv2 connection we need to
use the synchronous API.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
5f1ed29d by Tim Beale at 2019-01-07T00:23:07Z
s3:pylibsmb: Don't return '.'/'..' in .list()

The source4 .list() API wasn't doing this. This makes source3 and source4
have *almost* equivalent functionality, so we can start usign the
source3 API in the tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6bcd64a4 by Tim Beale at 2019-01-07T00:23:08Z
s3:pylibsmb: Make s3 and s4 listings return the same dict

Make the python dictionary generated by the s3 .list() use the same keys
as the current source4 dict. The reason for using the source4 dict is
that other python code depends on these keys (e.g. ntacls.py), whereas
the source3 API is currently unused.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
e4d1d535 by Tim Beale at 2019-01-07T00:23:08Z
s3:pylibsmb: Free async .list() memory

finfos was being allocated but never freed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
0af78faf by Tim Beale at 2019-01-07T00:23:08Z
s3:pylibsmb: Add .savefile() API to SMB py bindings

This provides a simple API for writing a file's contents and makes the
s3 API consistent with the s4 API.

All the async APIs here support SMBv2 so we don't need to use the sync
APIs at all.

Note that we have the choice here of using either cli_write_send() or
cli_push_send(). I chose the latter, because that's what smbclient uses.
It also appears to handle writing a large file better (i.e. one that
exceeds the max write size of the underlying connection).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
d7c7d620 by Tim Beale at 2019-01-07T00:23:08Z
s3:pylibsmb: Add .loadfile() API to SMB py bindings

Add a .loadfile API to read a file's contents. This provides a
convenient way to read a file and is consistent with the existing
source4 API, which is used by things like the GPO python code and the
ntacls backup.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
1d0d1a75 by Tim Beale at 2019-01-07T00:23:08Z
s3:libsmb: Avoid duplicated code by making cli_read_sink() public

cli_read_sink() and pull_helper() were essentially identical. By making
cli_read_sink() non-static, we can delete the latter.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
ea00215d by Tim Beale at 2019-01-07T00:23:08Z
s3:pylibsmb: Minor refactor to py_cli_list() variables

Add a define for the file attribute mask (we'll reuse it in a subsequent
patch), and make the variable type explicit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9a3e640b by Tim Beale at 2019-01-07T00:23:08Z
s3:pylibsmb: Add .deltree() API to SMB py bindings

This basically re-uses the underlying functionality of existing APIs in
order to support a .deltree() API, i.e.
- we use the .list() functionality (i.e. do_listing()) to traverse every
  item in the given directory.
- we then use either .unlink() (i.e. unlink_file()) or .rmdir() (i.e.
  remove_dir()) to delete the individual item.
- sub-directories are handled recursively, by repeating the process.

Note that the .deltree() API is currently only really used for testing
(and deleting GPO files). So the recursion is never going to be
excessive.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c95a869e by Tim Beale at 2019-01-07T00:23:08Z
tests: Completely replace s4 connection in smb tests

This test now uses the s3 python bindings completely, so we can remove
the s4 connection.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7f731d76 by Tim Beale at 2019-01-07T00:23:08Z
tests: Avoid hardcoding domain in test

Currently the sysvol domain directory is hard-coded, so the tests can
only ever run on the ad_dc.

This patch makes things marginally better by using the REALM
environmental variable instead. This allows us to run it against other
testenvs (like the SMBv2-only restoredc).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
01079b50 by Tim Beale at 2019-01-07T03:29:51Z
tests: Run SMB Py bindings tests against testenv with SMBv1-disabled

Sanity-check that the SMBv2 connection actually works by running it
against a testenv with SMBv1 disabled.

I've dropped 'local' from the ad_dc target, because it shouldn't be
needed. We're trying to test the client-side SMB connection, so running
it without 'local' is probably a better test.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Mon Jan  7 04:29:51 CET 2019 on sn-devel-144

- - - - -
8662ff67 by Tim Beale at 2019-01-07T21:48:27Z
s3:pylibsmb: Rename libsmb_samba_internal Py bindings to libsmb

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
bd66d46a by Tim Beale at 2019-01-07T21:48:27Z
tests: Rename libsmb_samba_internal test to libsmb

So that it better matches the updated Python bindings name.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
27e2b5c6 by Tim Beale at 2019-01-07T21:48:28Z
s3:pylibsmb: Rename 'credentials' param to match s4

s4 just calls it 'creds'. Renaming it will make it easier to convert
existing SMB connections over to use the new bindings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
3ba32aa5 by Tim Beale at 2019-01-07T21:48:28Z
s3:pylibsmb: Add flags used by .list() to SMB Py bindings

These flags are exposed by the s4 code. Python code that calls .list()
checks the returned attribs/mode for the directory listing, e.g. to work
out whether something is a sub-directory:

  if item['attrib'] & libsmb.FILE_ATTRIBUTE_DIRECTORY...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
af1fe6fd by Tim Beale at 2019-01-07T21:48:29Z
python/ntacls: Convert ntacls to use s3 flags

This helper code is just using the flags defined by the Python bindings.
Convert it over to use s3 bindings instead of s4.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
5a6fcdd8 by Tim Beale at 2019-01-07T21:48:29Z
tests: Avoid hardcoding domain in GPO tests

The realm varies by testenv. Currently the GPO tests will only run on
the ad_dc testenv.

A slightly better solution is to use the REALM environmental variable,
so that the test can run on any testenv.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
888c6080 by Tim Beale at 2019-01-07T21:48:30Z
netcmd: Refactor duplicated SMB connect in GPO commands

Do the SMB connection in a single helper function.

Note: this highlights that perhaps we want all SMB connections to be
signed, but we can fix that up separately.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
24cfa0ba by Tim Beale at 2019-01-08T02:13:48Z
netcmd: Small refactor to SMB connection in domain backup

Rework the code so we only do this in one place.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Tue Jan  8 03:13:48 CET 2019 on sn-devel-144

- - - - -
facd6f2d by Aliaksei Karaliou at 2019-01-08T02:40:26Z
s4:torture:base:bench-readwrite: fix uninitialized memory causing segfault

Allocation of 'struct smb_composite_connect' happens without zeroing
so that smb_composite_connect_send() thinks that connection
already exists and fails on its handling.

Signed-off-by: Aliaksei Karaliou <akaraliou at panasas.com>
Reviewed-by: Douglas Bagnall <dbagnall at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fa945bc1 by Volker Lendecke at 2019-01-08T02:40:26Z
libsmb: Use strlcpy instead of StrnCpy

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e74ad1b7 by Volker Lendecke at 2019-01-08T02:40:26Z
nmbd: Use strlcpy instead of StrnCpy

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2cc2a753 by Volker Lendecke at 2019-01-08T02:40:26Z
smbd: Use strlcpy instead of StrnCpy

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
cc0664dd by Volker Lendecke at 2019-01-08T02:40:26Z
lib: Remove StrnCpy

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
12ffef9f by Volker Lendecke at 2019-01-08T02:40:27Z
auth3: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
af95db60 by Volker Lendecke at 2019-01-08T02:40:27Z
winbindd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6b8c7715 by Volker Lendecke at 2019-01-08T02:40:27Z
libcli: Simplify smblsa_lookup_name

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
08be989a by Volker Lendecke at 2019-01-08T02:40:27Z
smbclient: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c9036f25 by Volker Lendecke at 2019-01-08T02:40:27Z
libndr: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0d208f42 by Volker Lendecke at 2019-01-08T02:40:27Z
pydsdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a74946e3 by Volker Lendecke at 2019-01-08T02:40:27Z
winbind: Fix a typo

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
542da7e8 by Volker Lendecke at 2019-01-08T02:40:28Z
passdb: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
62bc70d1 by Volker Lendecke at 2019-01-08T02:40:28Z
ntpd: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
186fff5f by Volker Lendecke at 2019-01-08T02:40:28Z
auth4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6f4b2cc9 by Volker Lendecke at 2019-01-08T02:40:28Z
pysecurity: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f1e71a2c by Volker Lendecke at 2019-01-08T02:40:28Z
winbind4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b06f0912 by Volker Lendecke at 2019-01-08T02:40:28Z
libnet4: Use dom_sid_str_buf

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a2402f97 by Justin Stephenson at 2019-01-08T02:40:28Z
s3: net: Do not set NET_FLAGS_ANONYMOUS with -k

This affects net rpc getsid and net rpc changetrustpw commands.
This avoids an anonymous IPC connection being made when -k is used,
this only affects net rpc getsid and net rpc changetrustpw commands.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13726

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
99c030bc by Björn Jacke at 2019-01-08T02:40:29Z
waf: fix waf errors with sendfail detection on Solaris

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f23287bc by Björn Jacke at 2019-01-08T06:03:49Z
waf: fix tri-state of --with-sendfile-support being auto the default

--with-sendfile-support should be used automatically if it's found.

Configure now also fails reliably for all platforms when sendfile was
explicitly requested but failed to be detected.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9707

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Jan  8 07:03:49 CET 2019 on sn-devel-144

- - - - -
0e2e6352 by Volker Lendecke at 2019-01-08T08:32:10Z
selftest: Test sids-to-xids with one failing sid

Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>

Signed-off-by: Volker Lendecke <vl at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>

- - - - -
6f2ccb59 by Volker Lendecke at 2019-01-08T08:32:10Z
idmap_tdb: If one SID fails to map, try the rest

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
ee4a33a4 by Volker Lendecke at 2019-01-08T12:15:35Z
idmap: In _wbint_Sids2UnixIDs, pass back what we have

SOME_UNMAPPED does not mean that nothing worthwhile is in here. We
need to pass what we have.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Jan  8 13:15:35 CET 2019 on sn-devel-144

- - - - -
c57f3794 by Douglas Bagnall at 2019-01-08T22:55:31Z
python/tests/blackbox: check_exit_code() explains by default

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
8760168a by Douglas Bagnall at 2019-01-08T22:55:31Z
python/traffic: remove duplicate test file

The other one is python/samba/tests/blackbox/testdata/traffic-sample-very-short.model

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4c463334 by Douglas Bagnall at 2019-01-08T22:55:31Z
traffic replay: allow --random-seed=0

Zero is a valid seed.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
79e84598 by Douglas Bagnall at 2019-01-08T22:55:31Z
traffic_replay: drop summary replay

The traffic_replay script has been able to replay a replay log as well
as a model, which was not used in practice and complicated the script.

If we want that feature, we can make a new script for it.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
f6b91d5b by Douglas Bagnall at 2019-01-08T22:55:31Z
traffic: use default value for --duration

We could not do this when we replayed traffic summaries (as opposed to
models), but now this script does not do that.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3b3b4fa6 by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic_learner: return an error code

And use it in tests, rather than expecting exact strings.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b275baeb by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic_learner: use samba.logger, not print(file=sys.stderr)

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d1a1c5d6 by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic learner: avoid truncated output files on error

add_argument(type=argparse.FileType('w'), ...) will open the file
and leave it empty if the script fails.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
709d2c84 by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic: Packet.from_line classmethod uses cls var

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9da79b54 by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic: fix mk_masked_dir doc and comments

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
158f1728 by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic: use namedtuple for ConversationAccounts namespace class

Saving memory, which reduces fork overhead.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b77ee628 by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic: delete empty temp directories

even if someone asked to keep the temporary data, they don't want
to see 5000 empty directries. Non-empty directories will remain.


Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
4f7ae5d9 by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic: make random_colour_print more deterministic, silent on demand

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
24d09d1d by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic: give Packet __slots__ for reduced memory

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
e866782a by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic: split is_a_real_packet() function out of class

So we can use it to determine whether a packet should be a Packet before
making the leap.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a352060f by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic: initialise conversation specific lists after the fork

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
a430b11c by Douglas Bagnall at 2019-01-08T22:55:32Z
traffic: rename packet_rate -> replay_speed for accuracy and room

We are soon going to have a self.packet_rate, and replay_speed is more
accurate in this case.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
ef747953 by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: Conversation gets given id, list of short packets

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
273eb3df by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: fix hash non-determinism when loading JSON

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Tim Beale <timbeale at samba.org>

- - - - -
7edf58dc by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: new version of model with packet_rate, version number

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
7b03e81c by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: generate sparser descriptions of conversations

Rather than building all the packets at this point, we stick to the
barest details of the packets (which is all the model gives us
anyway).

The advantage is that will take a lot less memory, which matters
because this process forks into many clients that were sharing and
mutate the conversation list.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c4d5bb59 by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: rework conversation generation to better use memory

Use less memory altogether and don't allocated shared mutable before
the fork.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
6268effb by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: avoid generating conversations without packets

Some "packets" don't generate any actual traffic. If we have a
conversation consisting only of those, we can avoid forking a client
for it.

This *slightly* increases the load over that which would be generated
otherwise for a given traffic rate, but that's OK.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
35bc6ee0 by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: reinitialise RNG in each child

Otherwise they all replay using the same random sequence.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
22cba701 by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: evoke OS error codes by name, not number

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b737552e by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic_replay: --stop-on-any-error option to not ignore client trouble

Sometimes you want to know if any client is crashing for any reason.
In those times use --stop-on-any-error for an early exit.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
17579dd8 by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: traffic_replay --latency-timeout to control final wait

Conversations that haven't finished within some acceptable margin of
on-time can be said to have failed. This is where you specify that
margin.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
65c02fd6 by Douglas Bagnall at 2019-01-08T22:55:33Z
traffic: assign context domain at start

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
41b0cfce by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic: simplify tests for badpassword_frequency

x <= 0 will fail one or both of the other test clauses.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
31c0809e by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic: avoid bare except: clauses

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
d727dcc8 by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic: generate more statistics in replay

Add more "header" values indicating the progress of the run as a
whole.

The new fields are:

Max sleep miss            - the longest sleep() oversleep. Indicates client load.
Maximum lag               - the longest gap between a planned packet
                            time and its actual time.
Start lag                 - the longest gap between intended and actual
                            conversation start.
Planned conversations     - how many conversations we meant to have.
Planned packets           - how many "packets" we thought we were making. Not
                            all "packets" result in actual operations or packets.
Unfinished conversations  - how many conversations had not finished
                            when they were killed.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
db510043 by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic: remove useless branch in stats report

This completes the work of 68c64c634ad7576b2ab5ef838c203320afd90f44,
but differs from that in that it makes no actual change because isatty
was not being called so was always evaluated as true.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b5d493f9 by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic: Conversation.add_short_packet is discerning about packets

If the packets really wouldn't do anything, we might as well not add them.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
9e0effc1 by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic: improve debug messages in traffic_replay

tracebacks and less nonsense at higher debug levels.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
bda7f35a by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic_model: don't report generation errors as parse errors

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
524777e6 by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic: add option to reanimate dying conversations

The traffic model is generated from a window in time, which makes
conversations appear to start and stop unnaturally at the window
boundaries. When the window is short compared to the traffic replay
time and the true expected conversation length, this has a significant
distorting effect, leading to more conversations than would be
expected to generate a given number of packets.

To offset this slightly we add the --conversation-persistence option
which tries to convert apparent death into a longish wait.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
affaeb94 by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic_replay: --old-scale to mimic the old traffic_replay

traffic_replay had a broken sense of traffic scale. That is fixed, but
in order to compare old and new tests, it helps to be able to
approximate the old behaviour.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
71cb2605 by Douglas Bagnall at 2019-01-08T22:55:34Z
python/tests: add TestCaseInTempdir.mktemp()

This gives you a name of a temporary file within the test case's tempdir.
Use it like this:

  with self.mktemp() as filename:
     self.check_run("samba-tool foo --output %s" % filename)
     self.assertStringsEqual(open(filename).read(), expected)

and filename will flick out of existence when the with block ends.

This is based on an idea used in the traffic_runner tests, which will
soon be adapted to use this method.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
daf039af by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic tests: use TestCaseInTempdir.mktemp()

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
c672a922 by Douglas Bagnall at 2019-01-08T22:55:34Z
traffic: assert json equality in learner test

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3c10ceca by Douglas Bagnall at 2019-01-08T22:55:35Z
traffic_replay: use packets per second as primary scale

The old -S/--scale-traffic is relative to the original model, which made
its relationship to true traffic volumes quite opaque

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
b2a9d4c1 by Douglas Bagnall at 2019-01-09T02:53:58Z
xml_docs: update traffic script documentation

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Tim Beale <timbeale at samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Wed Jan  9 03:53:58 CET 2019 on sn-devel-144

- - - - -
84069c8a by Joe Guo at 2019-01-09T09:15:12Z
netcmd/user: python[3]-gpgme unsupported and replaced by python[3]-gpg

python[3]-gpgme is deprecated since ubuntu 1804 and debian 9.
use python[3]-gpg instead, and adapt the API.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13728

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
a9def5c6 by Tim Beale at 2019-01-09T13:30:31Z
s3:libsmb: Revert SMB Py bindings name back to libsmb_samba_internal

In order to make it clear that the APIs in these Python bindings are
unstable and should not be used by external consumers, this patch
changes the name of the Python bindings back to libsmb_samba_internal.

To make the Python code that uses these bindings (i.e. samba-tool, etc)
look a little cleaner, we can just change the module name as we import
it, e.g.

  from samba.samba3 import libsmb_samba_internal as libsmb

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Wed Jan  9 14:30:31 CET 2019 on sn-devel-144

- - - - -
78f51a1d by Justin Stephenson at 2019-01-09T17:40:44Z
s3:libsmb: Check disable_netbios in socket connect

If the disable_netbios option is set then return NT_STATUS_NOT_SUPPORTED
for a port 139 connection in the low level socket connection code.

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Noel Power <nopower at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
499f051c by Justin Stephenson at 2019-01-09T17:40:44Z
s3:libsmb: Print debug message about Netbios

With a preceding patch, cli_connect_nb() will return
NT_STATUS_NOT_SUPPORTED when 'disable netbios' is set in smb.conf.

Print an informative error message to indicate Netbios is disabled
if this occurs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13727

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Noel Power <nopower at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ecbb2f78 by Justin Stephenson at 2019-01-09T17:40:44Z
s3:smbpasswd: Print debug message about Netbios

With a preceding patch, cli_connect_nb() will return
NT_STATUS_NOT_SUPPORTED when 'disable netbios' is set in smb.conf.

Print an informative error message to indicate Netbios is disabled
if this occurs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13727

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Noel Power <nopower at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
08867de2 by Justin Stephenson at 2019-01-09T21:38:21Z
s3:utils:net: Print debug message about Netbios

With a preceding patch, cli_connect_nb() will return
NT_STATUS_NOT_SUPPORTED when 'disable netbios' is set in smb.conf.

Print an informative error message to indicate Netbios is disabled
if this occurs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13727

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Noel Power <nopower at suse.com>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Wed Jan  9 22:38:21 CET 2019 on sn-devel-144

- - - - -
bf229de7 by Tim Beale at 2019-01-09T21:39:26Z
libcli: Add error log if insufficient SMB2 credits

Although it's unusual to hit this case, I was seeing it happen while
working on the SMB python bindings. Even with debug level 10, there was
nothing coming out to help pin down the source of the
NT_STATUS_INTERNAL_ERROR.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13736

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
fd355dff by Tim Beale at 2019-01-10T01:40:16Z
s3:libsmb: cli_smb2_list() can sometimes fail initially on a connection

cli_smb2_list() appears to be a slightly unique SMB operation in that it
specifies the max transaction size for the response buffer size. The
Python bindings highlighted a problem where if cli_smb2_list() were one
of the first operations performed on the SMBv2 connection, it would fail
due to insufficient credits. Because the response buffer size is
(potentially) so much larger, it requires more credits (128) compared
with other SMB operations.

When talking to a samba DC, the connection credits seem to start off at
1, then increase by 32 for every SMB reply we receive back from the
server. After cli_full_connection(), the connection has 65 credits. The
cli_smb2_create_fnum() in cli_smb2_list() adds another 32 credits, but
this is still less than the 128 that smb2cli_query_directory() requires.

This problem doesn't happen for smbclient because the cli_cm_open() API
it uses ends up sending more messages, and so the connection has more
credits.

This patch changes cli_smb2_list(), so it requests a smaller response
buffer size if it doesn't have enough credits available for the max
transaction size. smb2cli_query_directory() is already in a loop, so it
can span multiple SMB messages if for some reason the transaction size
isn't big enough for the listings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13736

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Thu Jan 10 02:40:16 CET 2019 on sn-devel-144

- - - - -
f1ff4075 by Volker Lendecke at 2019-01-10T01:41:29Z
dns: Do NULL checks in dns_sign_tsig

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
78214d3c by Volker Lendecke at 2019-01-10T01:41:29Z
dsdb: Print strerror in addition to errno

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6e0acc7c by Volker Lendecke at 2019-01-10T01:41:29Z
selftest: Remove duplicate dependency definition

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e29cb509 by Swen Schillig at 2019-01-10T01:41:29Z
Fix typos and ban the rams from samba

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <Volker.Lendecke at SerNet.DE>

- - - - -
08fd0561 by Swen Schillig at 2019-01-10T05:14:27Z
Fix typo and remove the sun and the clouds

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <Volker.Lendecke at SerNet.DE>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Jan 10 06:14:27 CET 2019 on sn-devel-144

- - - - -
7a5b65c8 by Ladislav Michl via samba-technical at 2019-01-10T05:15:21Z
lib: replace: snprintf: Whitespace clean up

Remove spaces before tab, spaces at the end of line and white
chars on blank line.

Signed-off-by: Ladislav Michl <ladis at linux-mips.org>
Reviewed-by: Björn Jacke <bjacke at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

- - - - -
d7d3ee18 by Björn Jacke at 2019-01-10T08:40:06Z
statvfs: fix bsize and frsize mixup

the block size (the real one) is the "fundamental file system block size" and
that is the frsize struct member in the statvfs struct. The bsize struct member
of the statvfs struct is *different* from the same named one of the statfs
struct.

See also http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_statvfs.h.html

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11810

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Thu Jan 10 09:40:06 CET 2019 on sn-devel-144

- - - - -
36ff1a51 by Tim Beale at 2019-01-10T15:21:23Z
netcmd: Fix broken samba-tool gpo commands

samba-tool gpo create|fetch|backup commands throw exceptions due to a
missing Python import:

ERROR(<class 'NameError'>): uncaught exception - name 'tempfile' is not
defined
  File "bin/python/samba/netcmd/__init__.py", line 184, in _run
    return self.run(*args, **kwargs)
  File "bin/python/samba/netcmd/gpo.py", line 980, in run
    tmpdir, gpodir = self.construct_tmpdir(tmpdir, gpo)
  File "bin/python/samba/netcmd/gpo.py", line 386, in construct_tmpdir
    tmpdir = tempfile.mkdtemp()

Introduced by commit e3320b6d3df7a0c7b refactor.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Thu Jan 10 16:21:23 CET 2019 on sn-devel-144

- - - - -
c53e8de4 by Björn Jacke at 2019-01-10T21:46:11Z
waf: check for utmpx struct member ut_host

BUG: https://bugzilla.samba.org/show_bug.cgi?id=2489

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
7643ee2e by Björn Jacke at 2019-01-10T21:46:11Z
s3/smbd: fix utmp hostname logging on Solaris

BUG: https://bugzilla.samba.org/show_bug.cgi?id=2489

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
121db8c0 by Björn Jacke at 2019-01-10T21:46:11Z
waf:lib/replace: fix a build error with non-gcc compilers

3a175830e579ab10231439657b23733338cd5634 added that variable which should have
been an empty initialization here. -Wno-format-zero-length (which might be
unsupported by the compiler) should not be set if we really only want to
initialize the cflags.

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
93c54a4b by Björn Jacke at 2019-01-11T01:53:57Z
statvfs: rename linux_statvfs to posix_statvfs

the statvfs call is posix standard and not Linux specific

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Björn Jacke <bjacke at samba.org>
Autobuild-Date(master): Fri Jan 11 02:53:57 CET 2019 on sn-devel-144

- - - - -
ef379880 by Aaron Haslett at 2019-01-11T01:55:01Z
dns: changing onelevel search for wildcard to subtree

SCOPE_ONELEVEL is used on wildcard dns searches, but onelevel searches
currently have a performance problem related to GUID indexing, so this
patch changes the search scope to SCOPE_SUBTREE.
In this case, as the onelevel and subtree sets of records are roughly
the same, and the query is matching against the DN itself, we don't
believe there's any benefit in using SCOPE_ONELEVEL over SCOPE_SUBTREE.

The onelevel performance problem will be fixed separately later, but in
the meantime this solves the DNS performance problem.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13738

Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

- - - - -
8af4ec75 by Volker Lendecke at 2019-01-11T05:01:01Z
selftest: Improve an error message

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Jan 11 06:01:01 CET 2019 on sn-devel-144

- - - - -
ad04a6ce by Ralph Boehme at 2019-01-11T22:11:11Z
s3:lib: add root_unix_token()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
92c2ed4d by Stefan Metzmacher at 2019-01-11T22:11:11Z
s3:lib: add per_thread_cwd_{check,supported,disable,activate}() helper functions

Only Linux it's possible to have a per thread current working directory
using unshare(CLONE_FS).

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
16166542 by Stefan Metzmacher at 2019-01-11T22:11:11Z
s3:smbd: prepare the usage of per_thread_cwd_*() functions

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
a62bc3f2 by Stefan Metzmacher at 2019-01-11T22:11:11Z
s3:torture: call per_thread_cwd_check() in vfstest.c main()

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
7f7ce0ec by Ralph Boehme at 2019-01-11T22:11:11Z
s3:smbd: let SMB_VFS_GETXATTRAT_SEND() do explicit impersonation

SMB_VFS_GETXATTRAT_SEND() gets passed a raw event context and the
default implementation uses that as well a raw threadpool. Impersonation
is done explicitly instead of by the tevent and pthreadpool wrappers.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
56dee840 by Ralph Boehme at 2019-01-11T22:11:11Z
s3:smbd: pass (raw) ev to SMB_VFS_GET_DOS_ATTRIBUTES_SEND() instead of smb_vfs_ev_glue

This also removes smb_vfs_ev_glue_[push|pop]_use() as the only caller
got removed.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
9f3d9ba4 by Ralph Boehme at 2019-01-11T22:11:11Z
s3:smbd: pass (raw) ev to dos_mode_at_send() instead of smb_vfs_ev_glue

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
edb75eeb by Ralph Boehme at 2019-01-11T22:11:12Z
s3:smbd: pass (raw) ev to fetch_dos_mode_send instead of smb_vfs_ev_glue

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
57c7aaa3 by Ralph Boehme at 2019-01-11T22:11:12Z
Revert "pthreadpool: reset monitor_fd after calling tevent_fd_set_auto_close()"

This reverts commit f68b5ee75f002ed542a8423070fb01e3b0e83f65.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
a75f5de1 by Ralph Boehme at 2019-01-11T22:11:12Z
Revert "pthreadpool: ignore the return value of poll(NULL, 0UL, 1)"

This reverts commit 6da0d68f49bbd82f5a08427779c9f5ebd6f755aa.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
96332ed1 by Ralph Boehme at 2019-01-11T22:11:12Z
Revert "pthreadpool: we need to use pthreadpool_tevent_per_thread_cwd() on the callers pool"

This reverts commit ff863f2d98ac5e12073af824b794404c3d7198c5.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
56f76944 by Ralph Boehme at 2019-01-11T22:11:12Z
Revert "vfs_aio_pthread: use event context and threadpool from user_vfs_evg"

This reverts commit cd37badc02b3e56f974168b28447444cd54ee541.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
29dd6f3e by Ralph Boehme at 2019-01-11T22:11:12Z
Revert "s3: vfs: add user_vfs_evg to connection_struct"

This reverts commit 2dd95c1c38b9e1ce32d3d1081b6ec177910087a4.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
cedbfbd9 by Ralph Boehme at 2019-01-11T22:11:12Z
Revert "s3: vfs: add smb_vfs_ev_glue"

This reverts commit 1251a536df4b1df58d9ddacab03d3ebe6f4e5b60.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8074922c by Ralph Boehme at 2019-01-11T22:11:12Z
Revert "smbd: introduce sconn->sync_thread_pool"

This reverts commit 0c97226356f2ba5f01a58d361371055caf11e2a7.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
29fc7c7d by Ralph Boehme at 2019-01-11T22:11:13Z
smbd: rename sconn->raw_thread_pool to sconn->pool

This is basically a revert of commit
2be7518ae5a3c046f5fca04ecc83f9f7044eac74.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
d032210d by Ralph Boehme at 2019-01-11T22:11:13Z
Revert "pthreadpool: test cancelling and freeing jobs of a wrapped pthreadpool_tevent"

This reverts commit fb6b6cf3e43165ced4b1039f2683d19f277c0792.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
9b7d2257 by Ralph Boehme at 2019-01-11T22:11:13Z
Revert "pthreadpool: implement pthreadpool_tevent_wrapper_create() infrastructure"

This reverts commit f9745d8b5234091c38e93ed57a255120b61f3ad7.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
67015e13 by Ralph Boehme at 2019-01-11T22:11:13Z
Revert "pthreadpool: add pthreadpool_restart_check[_monitor_{fd,drain}]()"

This reverts commit 3c4cdb290723432b00ff9ff88b892cb4e66e76cd.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
e94cd84b by Ralph Boehme at 2019-01-11T22:11:13Z
Revert "pthreadpool: call unshare(CLONE_FS) if available"

This reverts commit 65e4742d168454df6507d9e74993749063435dd6.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
f4d6c48c by Ralph Boehme at 2019-01-11T22:11:13Z
Revert "pthreadpool: add tests for pthreadpool_tevent_[current_job_]per_thread_cwd()"

This reverts commit fbafdc99ef2cef11a1a28e795ffe965cb53ef7fa.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
622ba519 by Ralph Boehme at 2019-01-11T22:11:13Z
Revert "pthreadpool: add pthreadpool_tevent_[current_job_]per_thread_cwd()"

This reverts commit 12a45ee1a66379ba7562729b835ce0e2e4bfb3b3.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
7fd9bc3b by Ralph Boehme at 2019-01-11T22:11:14Z
Revert "pthreadpool: test cancelling and freeing pending pthreadpool_tevent jobs/pools"

This reverts commit 40d15260d24d0071732f47873f395fce29b8a6f4.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
42e2ab7e by Ralph Boehme at 2019-01-11T22:11:14Z
Revert "pthreadpool: add a comment about a further optimization in pthreadpool_tevent_job_destructor()"

This reverts commit f23cac39b36b026650e0922c78fe0fd3fe567e35.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
7882941b by Ralph Boehme at 2019-01-11T22:11:14Z
Revert "pthreadpool: maintain a list of job_states on each pthreadpool_tevent_glue"

This reverts commit aa9b64eccfd037941512bad108c4e3946714a502.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
d818e511 by Ralph Boehme at 2019-01-11T22:11:14Z
Revert "pthreadpool: add helgrind magic to PTHREAD_TEVENT_JOB_THREAD_FENCE_*()"

This reverts commit 9b73fda926eb8493e80012794483039be66d4e6c.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
991ca9b5 by Ralph Boehme at 2019-01-11T22:11:14Z
Revert "pthreadpool: add some lockless coordination between the main and job threads"

This reverts commit 9656b8d8ee11ee351870286f16ea8fbe49112292.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
9e4bc1f4 by Ralph Boehme at 2019-01-11T22:11:14Z
Revert "pthreadpool: maintain a global list of orphaned pthreadpool_tevent_jobs"

This reverts commit 25756425aaf5465e56ea809cd415b6a387848919.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
44900b04 by Ralph Boehme at 2019-01-11T22:11:14Z
Revert "pthreadpool: add pthreadpool_tevent_job_cancel()"

This reverts commit 791c05144ee9296024cc0fdebe4afeaaf67e26bc.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
8e03cbe8 by Ralph Boehme at 2019-01-11T22:11:14Z
Revert "pthreadpool: split out pthreadpool_tevent_job from pthreadpool_tevent_job_state"

This reverts commit 245d684d28dab630f3d47ff61006a1fe3e5eeefa.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
718cc7a0 by Ralph Boehme at 2019-01-11T22:11:15Z
Revert "smbd: remove unused change_to_root_user() from brl_timeout_fn()"

This reverts commit c310647e560abc2343ef2ff0ab15daab0fba2650.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
3747dcb4 by Ralph Boehme at 2019-01-11T22:11:15Z
Revert "smbd: remove unused change_to_root_user() from smbd_sig_hup_handler()"

This reverts commit d0b1f96f086bdd613644f64b3f75ee8c4388f674.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
ac17919a by Ralph Boehme at 2019-01-11T22:11:15Z
Revert "smbd: avoid explicit change_to_user() in defer_rename_done() already done by impersonation"

This reverts commit e37e41b3cac52e3623f0c79f83733a51edb35c10.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
864e9851 by Ralph Boehme at 2019-01-11T22:11:15Z
Revert "smbd: implement smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers"

This reverts commit 1b804f7ae23f54a6c1004e5ff17b9df1376b5adb.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
97a7f886 by Ralph Boehme at 2019-01-11T22:11:15Z
Revert "smbd: make use of smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers"

This reverts commit 0dcaa0707bad67f7bfaa10ccaf167bfefbe87a0c.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
e717c903 by Ralph Boehme at 2019-01-11T22:11:15Z
Revert "smbd: add simple noop smbd_impersonate_{conn_vuid,conn_sess,root,guest}_create() wrappers"

This reverts commit 5285966e67cbee8519015df12a15e938e85e6ee7.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
3aaf7c14 by Ralph Boehme at 2019-01-11T22:11:15Z
Revert "smbd: add smbd_impersonate_debug_create() helper"

This reverts commit 23319ef5a2eff4811b685d4ab54179efc49bac99.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
78fa4408 by Ralph Boehme at 2019-01-11T22:11:15Z
Revert "smbd: add [un]become_guest() helper functions"

This reverts commit 7b5a47b84696e8e5c26207bd398742b883e598c2.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
0bd10a48 by Ralph Boehme at 2019-01-11T22:11:16Z
Revert "s4:messaging: make sure only imessaging_client_init() can be used with a wrapper tevent_context wrapper"

This reverts commit e186d6a06b1b300256a2cb4138f0532d518d0597.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
e2a5272a by Ralph Boehme at 2019-01-11T22:11:16Z
Revert "s3:messages: allow messaging_filtered_read_send() to use wrapper tevent_context"

This reverts commit 2b05f1098187e00166649c8ea7c63e6901b9d242.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
26107832 by Ralph Boehme at 2019-01-11T22:11:16Z
Revert "s3:messages: allow messaging_dgm_ref() to use wrapper tevent_context"

This reverts commit 9dc332060cf5f249ea887dbc60ec7a39b6f91120.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
1c3676f3 by Ralph Boehme at 2019-01-11T22:11:16Z
Revert "s3:messages: allow messaging_{dgm,ctdb}_register_tevent_context() to use wrapper tevent_context"

This reverts commit 660cf86639753edaa7a7a21a5b5ae207ae7d4260.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
2a62a98f by Ralph Boehme at 2019-01-11T22:11:16Z
Revert "s3:messages: protect against usage of wrapper tevent_context objects for messaging"

This reverts commit 7f2afc20e1b6397c364a98d1be006377c95e4665.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
122e141b by Ralph Boehme at 2019-01-11T22:11:16Z
Revert "smbd: remove unused smbd_server_connection->ev_ctx"

This reverts commit 721fbbfa7001b2788602106101f0407483894322.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
88016bac by Ralph Boehme at 2019-01-11T22:11:16Z
Revert "smbd: add an effective {smb,smbd_smb2}_request->ev_ctx that holds the event context used for the request processing"

This reverts commit 894e5001c747ce765dad5517778dda55d7d1f4d9.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
de10c611 by Ralph Boehme at 2019-01-11T22:11:16Z
Revert "smbd: add an effective connection_struct->user_ev_ctx that holds the event context used for the current user"

This reverts commit f2df8be16be9dccd3d10ec060f1efbe5007a28c6.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
9d47128f by Ralph Boehme at 2019-01-11T22:11:17Z
Revert "smbd: use sconn->root_ev_ctx for smbd_sig_{term,hup}_handler()"

This reverts commit c835ffa72ddfd2431d22909148913b50f0d829d1.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
50995917 by Ralph Boehme at 2019-01-11T22:11:17Z
Revert "smbd: use sconn->root_ev_ctx for brl_timeout_fn()"

This reverts commit 182991c26c8149c79b13a277b9822efc49fd1df0.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
296a15a0 by Ralph Boehme at 2019-01-11T22:11:17Z
Revert "smbd: add smbd_server_connection->{root,guest}_ev_ctx pointer"

This reverts commit 146938217ed1ab9a7a9f38c055fec5513cbd5c4d.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
523a64e2 by Ralph Boehme at 2019-01-11T22:11:17Z
Revert "smbd: use raw_ev_ctx to clear the MSG_SMB_CONF_UPDATED registration"

This reverts commit f5f9b719741465c7be3de20a6a69ec106ecc4568.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
ac597b07 by Ralph Boehme at 2019-01-11T22:11:17Z
Revert "smbd: explain that/why we use the raw tevent_context for linux_oplock_signal_handler()"

This reverts commit dc517b20f60a156d73fdd551557eb1d4366dfdeb.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
5afd1919 by Ralph Boehme at 2019-01-11T22:11:17Z
Revert "smbd: explain that/why we use the raw tevent_context for do_break_to_none()"

This reverts commit 1d5210b615035e46b90758ac7aa4ceec9174bee5.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
4036fc89 by Ralph Boehme at 2019-01-11T22:11:17Z
Revert "smbd: explain that/why we use the raw tevent_context for oplock_timeout_handler()"

This reverts commit e73eaa3c8004d3d8aff316cdb26b0bef85eceaca.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
bcb4723a by Ralph Boehme at 2019-01-11T22:11:18Z
Revert "smbd: explain that/why we use the raw tevent_context for lease_timeout_handler()"

This reverts commit 52f098d38da72d6eff3c4cac61487da897a8651c.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
46d3a154 by Ralph Boehme at 2019-01-11T22:11:18Z
Revert "smbd: explain that/why we use the raw tevent_context for update_write_time_handler()"

This reverts commit 7cfafaf190643eb28fc95d21a02a4e5e529e16d1.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
6ec96ac7 by Ralph Boehme at 2019-01-11T22:11:18Z
Revert "vfs_glusterfs: explain that/why we use the raw tevent_context in init_gluster_aio()"

This reverts commit b0af5715b0a7592a9728e0b76f653b9ab6228708.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
5d8f495b by Ralph Boehme at 2019-01-11T22:11:18Z
s3:smbd: use passed in ev in smbd_smb2_query_directory_send()

The ev argument is also a raw ev.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
57e2fb01 by Ralph Boehme at 2019-01-11T22:11:18Z
s3:smbd: don't use async dosmode if per-thread cwd is not available

The additional overhead for the async processing should be avoided when
we know the lower level has to fallback to sync processing anyway.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
bc539d81 by Ralph Boehme at 2019-01-11T22:11:18Z
vfs_default: add sync fallback to pread_send/recv

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
4673a8b9 by Ralph Boehme at 2019-01-11T22:11:18Z
vfs_default: add sync fallback to pwrite_send/recv

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
2f406fee by Ralph Boehme at 2019-01-11T22:11:18Z
vfs_default: add sync fallback to fsync_send/recv

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
e2f46c93 by Ralph Boehme at 2019-01-11T22:11:19Z
vfs_aio_pthread: use struct initializer for aio_open_private_data

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
a307e798 by Ralph Boehme at 2019-01-11T22:11:19Z
vfs_aio_pthread: store conn instead of sconn in aio_open_private_data

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
47c443b4 by Ralph Boehme at 2019-01-11T22:11:19Z
vfs_aio_pthread: add sync fallback

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
dcd0edfd by Ralph Boehme at 2019-01-11T22:11:19Z
Revert "smbd: add smbd_server_connection->raw_ev_ctx pointer"

This reverts commit 6114f9545fa856717220658e87f2a60f6767b7f4.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

- - - - -
168079b2 by Stefan Metzmacher at 2019-01-12T02:12:09Z
tevent: deprecate tevent_wrapper api again

Samba doesn't use it anymore and we don't want to
invite new users of that api without further discussion.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Sat Jan 12 03:12:09 CET 2019 on sn-devel-144

- - - - -
a2f718fb by Swen Schillig at 2019-01-12T02:13:31Z
lib: remove unused function nttime_from_string()

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7817e42d by Stefan Metzmacher at 2019-01-12T02:13:31Z
librpc: add dcerpc_get_auth_length() helper function

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c919514d by Stefan Metzmacher at 2019-01-12T02:13:31Z
librpc: add dcerpc_get_auth_{type,level,context_id}() helper functions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a94c4e55 by Stefan Metzmacher at 2019-01-12T02:13:31Z
librpc: add SMB_ASSERT(blob->length >= DCERPC_NCACN_PAYLOAD_OFFSET) protection

A lot of functions rely on having the 16 bytes dcerpc header to operate
on. This makes it more obvious and makes sure they can't be misused in
future.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
36d5268a by Stefan Metzmacher at 2019-01-12T02:13:31Z
s4:torture: make test_GetUserName() more strict

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0b972ec6 by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server/samr: Fix the O3 developer build

Different gcc versions complain at different places.

I recently got these:

../source4/rpc_server/samr/dcesrv_samr.c: In function
‘dcesrv_samr_QueryDisplayInfo2’:
../source4/rpc_server/samr/dcesrv_samr.c:4117:2: error: ‘q.out.result.v’
may be used uninitialized in this function [-Werror=maybe-uninitialized]
  return result;
  ^
../source4/rpc_server/samr/dcesrv_samr.c: In function
‘dcesrv_samr_QueryDisplayInfo3’:
../source4/rpc_server/samr/dcesrv_samr.c:4151:2: error: ‘q.out.result.v’
may be used uninitialized in this function [-Werror=maybe-uninitialized]
  return result;
  ^
cc1: all warnings being treated as errors

../source4/rpc_server/samr/dcesrv_samr.c: In function
‘dcesrv_samr_QueryDisplayInfo3’:
../source4/rpc_server/samr/dcesrv_samr.c:4151:2: error: ‘q.out.result.v’
may be used uninitialized in this function [-Werror=maybe-uninitialized]
  return result;
  ^
In file included from ../source4/rpc_server/samr/dcesrv_samr.c:4447:0:
default/librpc/gen_ndr/ndr_samr_s.c: In function ‘samr__op_dispatch’:
default/librpc/gen_ndr/ndr_samr_s.c:597:18: error: ‘q.out.result.v’ may
be used uninitialized in this function [-Werror=maybe-uninitialized]
   r2->out.result = dcesrv_samr_QueryDisplayInfo2(dce_call, mem_ctx,
r2);
                  ^
cc1: all warnings being treated as errors

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f17cd478 by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server: remove dcerpc_remote dependency from dcerpc_server

dcerpc_remote is just an implementation and does not belong to the
core dcerpc server.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6739d1b8 by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server: let test_rpc_dns_server_dnsutils depend on dcerpc_server

The core dcerpc server is required here, which also implies
DCERPC_COMMON.

This is required to move common/reply.c dcesrv_auth.c from
DCERPC_COMMON to dcerpc_server in the next commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a71558cf by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server: move dcesrv_auth.c and common/reply.c to dcerpc_server

They belong to the core dcerpc server.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5f33379e by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server: remove unused dcesrv_connection->transport_session_key

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
961ebf22 by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server: add dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
707105d0 by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server/backupkey: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c989e35c by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server/dnsserver: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e1caa511 by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server/drsuapi: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
73bda4b6 by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server/lsa: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
aa6d9e49 by Stefan Metzmacher at 2019-01-12T02:13:32Z
s4:rpc_server/netlogon: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
89c8870c by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server/samr: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ba127cab by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server/srvsvc: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
014e55b8 by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server/winreq: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
905417c2 by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server/common: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
13c534bd by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server: add dcesrv_{auth,transport}_session_key() functions

They take struct dcesrv_call_state as argument and will replace
dcesrv_{inherited,fetch}_session_key() which take dcesrv_connection
as argument.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8a171c69 by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server/lsa: make use of dcesrv_transport_session_key()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dbb381ad by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server/samr: make use of dcesrv_transport_session_key()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2f5c344b by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server/drsuapi: make use of dcesrv_auth_session_key()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
851c06f4 by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server: allow a NULL function pointer in dcesrv_fetch_session_key()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6df71436 by Stefan Metzmacher at 2019-01-12T02:13:33Z
s4:rpc_server: split out a dcesrv_session_info_session_key() helper function

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d2aecd87 by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server: only AUTH_TYPE_NONE should get a transport session key

There's only a logic change for NCALRPC and NCACN_UNIX_STREAM.
dcesrv_generic_session_key() already returned
NT_STATUS_NO_USER_SESSION_KEY for all other cases.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
00e92045 by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server: make dcesrv_endpoint_connect() static

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e0a963a7 by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server: only setup the auth_state.session_key fn in dcesrv_endpoint_connect()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
57edc9a0 by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server: only pass dcesrv_auth to auth_state.session_key_fn()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b8eae043 by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server: remove unused dcesrv_inherited_session_key()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ace06729 by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server: merge dcesrv_fetch_session_key() into dcesrv_transport_session_key()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d1f5e5d5 by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server: add some const to dcesrv_handle_fetch()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
47a941ca by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server: add dcesrv_handle_{create,lookup}() functions

They take struct dcesrv_call_state as argument and will replace
dcesrv_handle_{new,fetch}() which take dcesrv_connection_context
as argument.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ec3e404a by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server/drsuapi: make use of dcesrv_handle_create()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
89a42fc0 by Stefan Metzmacher at 2019-01-12T02:13:34Z
s4:rpc_server/eventlog: make use of dcesrv_handle_create()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
884a18fa by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server/lsa: make use of dcesrv_handle_create()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
70a5d21d by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server/samr: make use of dcesrv_handle_create()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
86661aba by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server/winreg: make use of dcesrv_handle_create()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
237ae037 by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server: only use the new dcesrv_handle_{create,lookup}() api

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8639d730 by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server: remove the old dcesrv_handle_{new,fetch}() api

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
f6a29686 by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server/handles: make use dcesrv_call_session_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
62d45e16 by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server: add dcesrv_call_auth_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
10816794 by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server/samr: make use of dcesrv_call_auth_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fc596ef1 by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server/lsa: make use of dcesrv_call_auth_info()

It's enough to check the auth_type for DCERPC_AUTH_TYPE_SCHANNEL,
there's no need to also check the auth_level for integrity or privacy.

The gensec layer already required at least DCERPC_AUTH_LEVEL_INTEGRITY,
see schannel_update_internal().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
cd380d8a by Stefan Metzmacher at 2019-01-12T02:13:35Z
s4:rpc_server/netlogon: simplify logic of dcesrv_netr_creds_server_step_check()

It's enough to check the auth_type for DCERPC_AUTH_TYPE_SCHANNEL,
there's no need to also check the auth_level for integrity or privacy.

The gensec layer already required at least DCERPC_AUTH_LEVEL_INTEGRITY,
see schannel_update_internal().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6b49e325 by Stefan Metzmacher at 2019-01-12T02:13:36Z
s4:rpc_server/netlogon: make use of dcesrv_call_auth_info()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c621a873 by Stefan Metzmacher at 2019-01-12T02:13:36Z
s4:rpc_server/remote: make use of dcesrv_call_credentials()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
72643248 by Stefan Metzmacher at 2019-01-12T02:13:36Z
s4:rpc_server: use helper variables to access 'struct dcesrv_auth' in dcesrv_auth.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b8d38417 by Stefan Metzmacher at 2019-01-12T02:13:36Z
s4:rpc_server: use helper variables to access 'struct dcesrv_auth' in common/reply.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
93ae817a by Stefan Metzmacher at 2019-01-12T02:13:36Z
s4:rpc_server: use helper variables to access 'struct dcesrv_auth' in dcerpc_server.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9a3ea90c by Stefan Metzmacher at 2019-01-12T02:13:36Z
s4:rpc_server: introduce dcesrv_call_state->auth_state

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8ec93292 by Stefan Metzmacher at 2019-01-12T02:13:36Z
s4:rpc_server: make use of dcesrv_call_state->auth_state in dcesrv_auth.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
097299ae by Stefan Metzmacher at 2019-01-12T02:13:36Z
s4:rpc_server: make use of dcesrv_call_state->auth_state in common/reply.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
58021612 by Stefan Metzmacher at 2019-01-12T02:13:36Z
s4:rpc_server: make use of dcesrv_call_state->auth_state in dcerpc_server.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e2a05c3a by Stefan Metzmacher at 2019-01-12T02:13:37Z
s4:rpc_server: allocate struct dcesrv_auth with talloc

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2640f60e by Stefan Metzmacher at 2019-01-12T02:13:37Z
s4:rpc_server: split out dcesrv_auth_prepare_gensec()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
76976eab by Stefan Metzmacher at 2019-01-12T02:13:37Z
s4:rpc_server: add some protection checks to dcesrv_auth_prepare_gensec()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b46df3a8 by Stefan Metzmacher at 2019-01-12T02:13:37Z
s4:rpc_server: split out log_successful_dcesrv_authz_event()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0191516e by Stefan Metzmacher at 2019-01-12T02:13:37Z
s4:rpc_server: set auth_invalid = true on disconnect

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
13e52cc9 by Stefan Metzmacher at 2019-01-12T02:13:37Z
s4:rpc_server: replace dce_conn->allow_auth3 with auth->auth_started

auth3 is allowed if auth_started is true and auth_finished is false.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d8293acd by Stefan Metzmacher at 2019-01-12T02:13:37Z
s4:rpc_server: replace dce_conn->allow_request with auth->auth_finished

They both had the same lifetime and the disconnect case is now
caught by auth->auth_invalid = true.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6b02b94d by Stefan Metzmacher at 2019-01-12T02:13:37Z
s4:rpc_server: add a min_auth_level to context handles

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
38e0c06a by Stefan Metzmacher at 2019-01-12T02:13:37Z
s4:rpc_server: move bind_time_features to dcesrv_assoc_group

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c76a5be8 by Stefan Metzmacher at 2019-01-12T02:13:38Z
s4:rpc_server: add dcesrv_iface_state_{store,find}_{assoc,conn}() helpers

They can be used instead of
dcesrv_connection_context->private_data and
dcesrv_assoc_group->proxied_id.

This is the first step to hide internal details of
the core dcerpc server from the interface implementations.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5cbcc78a by Stefan Metzmacher at 2019-01-12T02:13:38Z
s4:rpc_server/netlogon: make use of dcesrv_iface_state_{create,find}_conn()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
245a0ef7 by Stefan Metzmacher at 2019-01-12T02:13:38Z
s4:rpc_server/dnsserver: make use of dcesrv_iface_state_{create,find}_conn()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
965abe8e by Stefan Metzmacher at 2019-01-12T02:13:38Z
s4:rpc_server/remote: introduce remote_get_private()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1f348d07 by Stefan Metzmacher at 2019-01-12T02:13:38Z
s4:rpc_server/remote: remote_op_bind already has the table available

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d9b88ccd by Stefan Metzmacher at 2019-01-12T02:13:38Z
s4:rpc_server/remote: reformat code to get "dcerpc_remote:binding"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
23d1393a by Stefan Metzmacher at 2019-01-12T02:13:38Z
s4:rpc_server/remote: reformat code to get "dcerpc_remote:use_machine_account"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3a17389e by Stefan Metzmacher at 2019-01-12T02:13:38Z
s4:rpc_server/remote: add dcerpc_remote:allow_anonymous_fallback option

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
954dd89f by Stefan Metzmacher at 2019-01-12T02:13:38Z
selftest: use "dcerpc_remote:allow_anonymous_fallback = yes" for rpc_proxy

This already uses anonymous credentials as the remote connection is done
during the bind before the realm authentication has started.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
199d4ba7 by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server/remote: fail the connection if the remote server don't support MPX

If we already negotiated with our client to support concurent multiplexing (MPX),
we need a remote connection that also supports it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a8134191 by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server/remote: defer the connect to the first client request

This means we have a chance to use delegated credentials from the client
if available, as the authentication is already completed.

Before we only ever used the transport inherited credentials
via SMB or anonymous.

Note: most times we still fallback to anonymous...

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9d387919 by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server/remote: introduce struct dcesrv_remote_call

This holds the state for async requests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9616f6b8 by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server/remote: make use dcesrv_iface_state_{store,find}_{assoc,conn}() helpers

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2ed7db44 by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server/winreg: don't cache an ldb connection per presentation context

We should do this per context handle, as these are available on all
connections of the association group.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
792b6b9d by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server: remove unused dcesrv_assoc_group->proxied_id

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
28f4cb44 by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server: remove unused dcesrv_connection_context->private_date

dcesrv_iface_state_{create,find}_{assoc,conn}() should be used instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
70b00c75 by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server: only use context within op_bind() hooks and dcesrv_interface_bind_*() functions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e9eb8e6a by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server: only pass context to op_bind() hooks

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6f53c993 by Stefan Metzmacher at 2019-01-12T02:13:39Z
s4:rpc_server: SMB_ASSERT(auth->auth_finished); in order to get auth details

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d47f9af7 by Stefan Metzmacher at 2019-01-12T02:13:40Z
py:dcerpc/raw_testcase: add pfc_flags_2nd and use_auth3 options to do_generic_bind()

This makes it more flexible and allows to write complex tests in an
easier fashion.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5466ed12 by Stefan Metzmacher at 2019-01-12T02:13:40Z
py:dcerpc/raw_protocol: add tests for delayed header signing activation

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
fb9f6133 by Stefan Metzmacher at 2019-01-12T02:13:40Z
py:dcerpc/raw_protocol: consistently call self.recv_pdu(timeout=0.01) after auth3

When we don't expect a FAULT, we should wait a little bit to check
there's no response to auth3 request. This reduces the raw_procol
test from 45s down to 35s total runtime against Windows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
975e8e1f by Stefan Metzmacher at 2019-01-12T02:13:40Z
s4:rpc_server: fix DCERPC_PFC_FLAG_SUPPORT_HEADER_SIGN negotiation to match Windows

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
aa3f1f86 by Stefan Metzmacher at 2019-01-12T02:13:40Z
py:dcerpc/raw_protocol: enable tests with the DCERPC_PFC_FLAG_SUPPORT_HEADER_SIGN bit

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
276286ca by Stefan Metzmacher at 2019-01-12T02:13:40Z
selftest:Samba4: allow dcerpc auth level connect:lsarpc = yes in chgdcpass

This is required to explore the details of security context multiplexing
using lsa_GetUserName().

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b1c6b954 by Stefan Metzmacher at 2019-01-12T02:13:40Z
py:dcerpc/raw_protocol: add test_assoc_group_ok2 to check assoc groups over ncacn_np

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c192dc1c by Stefan Metzmacher at 2019-01-12T02:13:40Z
s4:rpc_server: don't replace '\\pipe\\' with '\\PIPE\\'

This is not what Windows returns (at least for \\pipe\lsass).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a8feb556 by Stefan Metzmacher at 2019-01-12T02:13:40Z
py:dcerpc/raw_protocol: add test_assoc_group_fail3()

This demonstrates that assoc groups are only shared on the
same transport (endpoint).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b3659fb5 by Stefan Metzmacher at 2019-01-12T02:13:41Z
s4:rpc_server: only share assoc group ids on the same transport

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a650120d by Stefan Metzmacher at 2019-01-12T02:13:41Z
py:dcerpc/raw_protocol: demonstrate that \\pipe\lsarpc returns \\pipe\lsass as secondary_address

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
98d58722 by Stefan Metzmacher at 2019-01-12T02:13:41Z
s4:rpc_server: make it possible to specify ncacn_np_secondary_endpoint

Even a connect to \\pipe\lsarpc should return a secondary_address
of '\\pipe\\lsass'. But that will be implemented in a following commit.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4bb3a66a by Stefan Metzmacher at 2019-01-12T02:13:41Z
s4:rpc_server/lsa: specify \\pipe\lsass as ncacn_np_secondary_endpoint

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ae467704 by Stefan Metzmacher at 2019-01-12T02:13:41Z
s4:pyrpc: make use of pytalloc_get_type() in py_dcerpc_syntax_init_helper()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
8bd0f440 by Stefan Metzmacher at 2019-01-12T02:13:41Z
s4:pyrpc: add py_dcerpc_ndr_pointer_deref/wrap() infrastructure

Some idl files use more than one layer of unique pointers. e.g.

    NTSTATUS lsa_GetUserName(
            [in,unique] [string,charset(UTF16)] uint16 *system_name,
            [in,out,ref] lsa_String **account_name,
            [in,out,unique] lsa_String **authority_name
            );

In order to specify *io.in.authority_name = NULL,
we need to wrap the pointer value (lsa_String or None)
into an base.ndr_pointer() object.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
1d7930c8 by Stefan Metzmacher at 2019-01-12T02:13:41Z
pidl/Python: use py_dcerpc_ndr_pointer_wrap/deref if multiple pointer levels are used

This will help the raw_protocol test to explore lsa_GetUserName.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3f535ed1 by Stefan Metzmacher at 2019-01-12T02:13:41Z
py:dcerpc/raw_testcase: add assertEqualsStrLower()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a0b23063 by Stefan Metzmacher at 2019-01-12T02:13:41Z
py:dcerpc/raw_protocol: add tests to demonstrate how security context multiplexing works

Important things are this:
- It's not required to use the bind time feature negotiation in order
  to use it, it's only a hint for the client, but nothing is really
  negotiated, unlike the request multiplexing with the
  DCERPC_PFC_FLAG_CONC_MPX.
- There's special handling related to AUTH_LEVEL_CONNECT
  and requests without auth trailer
- An security context is identified by the unique
  tuple of auth_type, auth_level and auth_context_id (all together!),
  not just the auth_context_id.
- There's a limit of 2049 explicit authentication contexts.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
7bc6ec81 by Stefan Metzmacher at 2019-01-12T02:13:41Z
s4:rpc_server: implement security context multiplexing

There're some systems like Cisco ISE use security multiplexing
without checking (via bind time feature negotiation)
the server supports it.

Others like VMWare View, fallback to NT4 style netlogon
connections without using netlogon secure channel,
which then triggers an error, with "server schannel = yes",
see https://bugzilla.samba.org/show_bug.cgi?id=13464.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ea0a7041 by Stefan Metzmacher at 2019-01-12T02:13:42Z
selftest:Samba4: run the raw_protocol test with a limit of 8 auth contexts

This is much faster than exploring the limit of 2049 during autobuild.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e61f9406 by Stefan Metzmacher at 2019-01-12T05:25:37Z
selftest:Samba4: run fl2003dc without security context multiplexing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=7113
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11892

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Jan 12 06:25:37 CET 2019 on sn-devel-144

- - - - -
5232979d by Noel Power at 2019-01-12T23:40:26Z
python: Fix memory leak with ParseTuple (using 'es' format)

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
253af8b8 by Noel Power at 2019-01-12T23:40:26Z
python: Add new compat PYARG_STR_UNI format

In python2 PYARG_STR_UNI evaluates to et which allows str type
(e.g bytes) pass through unencoded and accepts unicode objects
encoded as utf8
In python3 PYARG_STR_UNI evaluates to es which allows str type
encoded as named/specified encoding

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
efc4570d by Noel Power at 2019-01-12T23:40:26Z
auth/credentials: use 'et' as format for ParseTuple with python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
a7993770 by Noel Power at 2019-01-12T23:40:26Z
python: use 'et' as format for ParseTuple with python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
b6c8ef5f by Noel Power at 2019-01-12T23:40:26Z
s4/libnet: use 'et' as format for ParseTuple with python2

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
f8758b3b by Noel Power at 2019-01-12T23:40:26Z
lib/ldb/tests/python: Add test to pass utf8 encoded bytes to ldb.Dn

This test should demonstrate an error with the 'es' format in python
where a 'str' byte-string is passed (containing utf8 encoded bytes)
with some characters that cannot be decoded as ascii. The same
code if run in python3 should generate an error (needs string not
bytes)

Also Add knownfail for ldb.Dn passed utf8 encoded byte string
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
45f5337f by Noel Power at 2019-01-12T23:40:26Z
selftest: Enable ldb.python for PY3

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
8900e0b4 by Noel Power at 2019-01-13T02:53:00Z
lib/ldb: Use new PYARG_ES format for parseTuple

While 'es' format works great for unicode (in python2) and
str (in python3) The behaviour with str (in python2) is unexpected.
In python2 the str type is (re-encoded) with the specified encoding.
In python2 the 'et' type would be a better match, that ensures 'str'
type is treated like it was with 's' (no reencoding) and unicode is
encoded with the specified encoding. However in python3 'et' allows
byte (or bytearray) params to be accepted (with no reencoding), we
don't want this. This patch adds a new PYARG_STR_UNI format code which
is a hybrid, in python2 it evaluates to 'et' and in python3 'es' and
so gives the desired behaviour for each python version.

Additionally remove the associated known fail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13616
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Sun Jan 13 03:53:00 CET 2019 on sn-devel-144

- - - - -
bae41545 by Tim Beale at 2019-01-14T02:30:20Z
s3:pylibsmb: Add .get_acl() API to SMB py bindings

There is no obvious async-equivalent of cli_query_security_descriptor(),
so it will throw an error if anyone tries to use it in multi-threaded
mode. Currently only samba-tool and tests use the (s4) .get_acl() API,
both of which will be fine using the synchronous API.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
15013ff9 by Tim Beale at 2019-01-14T02:30:20Z
tests: Change ntaclsbackup tests over to use s3 Py bindings

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

- - - - -
792b8d3f by Tim Beale at 2019-01-14T05:49:09Z
netcmd: Change domain backup commands to use s3 SMB Py bindings

This means we can now backup a DC that has SMBv1 disabled.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Mon Jan 14 06:49:09 CET 2019 on sn-devel-144

- - - - -
3672f02e by Ralph Boehme at 2019-01-14T18:13:14Z
vfs: use struct initializer in smb_vfs_call_get_dos_attributes_send

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
2720d95d by Ralph Boehme at 2019-01-14T18:13:14Z
vfs: perform impersonation in smb_vfs_call_get_dos_attributes_done()

This is needed as the callback might be called in an arbitrary
impersonation state.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
333a180b by Ralph Boehme at 2019-01-14T18:13:14Z
vfs: use struct initializer in smb_vfs_call_getxattrat_send()

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
0b7c6d50 by Ralph Boehme at 2019-01-14T18:13:14Z
vfs: perform impersonation in smb_vfs_call_getxattrat_done()

This is needed as the callback might be called in an arbitrary
impersonation state.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
d21be78e by Ralph Boehme at 2019-01-14T18:13:15Z
vfs_default: use change_to_user_by_fsp() instead of change_to_user()

Cosmetic change.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
75e2e242 by Ralph Boehme at 2019-01-14T18:13:15Z
s3:smbd: perform impersonation in dos_mode_at_vfs_get_dosmode_done()

This is needed as the callback might be called in an arbitrary
impersonation state.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
c6728403 by Ralph Boehme at 2019-01-14T18:13:15Z
s3:smbd: perform impersonation in smb2_query_directory_dos_mode_done()

This is needed as the callback might be called in an arbitrary
impersonation state.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
6619cec3 by Ralph Boehme at 2019-01-14T21:30:24Z
s3:smbd: perform impersonation in smb2_query_directory_fetch_write_time_done()

This is not strictly required, as we ne never trigger additional VFS
requests via this codepath. But for safety reasons ensure we're running
in the correct impersonation state.

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Mon Jan 14 22:30:24 CET 2019 on sn-devel-144

- - - - -
12398a2d by Ralph Wuerthner at 2019-01-15T03:13:15Z
vfs_fileid: fix get_connectpath_ino

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13741

Signed-off-by: Ralph Wuerthner <ralph.wuerthner at de.ibm.com>
Reviewed-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Tue Jan 15 04:13:15 CET 2019 on sn-devel-144

- - - - -
a7e28a7e by Volker Lendecke at 2019-01-15T06:53:21Z
addns: Centralize siteless lookup fallback

We had the same logic 3 times, coalesce into one

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
ff2dbe24 by Volker Lendecke at 2019-01-15T06:53:21Z
dns_update: samba_dnsupdate's exit code is not an errno

This avoids confusing messages, samba_dnsupdate returns the number of
failed updates

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
e9e4aeaf by Volker Lendecke at 2019-01-15T06:53:21Z
libcli/dns: Add resolv.conf parsing

Right now this only looks at the nameserver setting. It is initally made for
asynchronous AD DC lookup routines, where we don't need the "search", "domain"
and other settings. When we convert general "net", "smbclient" and others to
use this, we might either add "domain" handling to this code or look at
something like c-ares which already does it.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5f393deb by Volker Lendecke at 2019-01-15T06:53:21Z
libcli/dns: Add dns_lookup

Wrapper function to parse resolv.conf and talk to multiple nameservers. This is
the code where we might want to add a "working nameserver" cache. glibc always
looks at the first configured nameserver. If that's dead, glibc runs into a
timeout and only then asks the second one that might succeed. When more than
one dns query is to be performed, these timeouts add up.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
b7418203 by Volker Lendecke at 2019-01-15T06:53:21Z
dns_lookup: Let make test override the resolv.conf location

Make this a separate commit: That is the feature that libc unfortunately does
not give us.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
237c06aa by Volker Lendecke at 2019-01-15T06:53:21Z
libcli/dns: Make "clidns" a library

This will be linked into the SAMBA_LIBRARY "addns" in the next step. Because
the other user, "dnsserver_common", is also a library, we can't link this as a
subsystem anymore.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
dbbce1a4 by Volker Lendecke at 2019-01-15T06:53:21Z
libcli/dns: clidns must depend on ndr_standard, not on NDR_DNS

Otherwise we can't link this into other libraries

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a8191f88 by Volker Lendecke at 2019-01-15T06:53:22Z
libcli/dns: Add dns_res_rec_get_sockaddr

Pull the address from a res_rec if it's there

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
61e64791 by Volker Lendecke at 2019-01-15T06:53:22Z
selftest: add central dns forwarder

This is a small DNS server that has hard redirects to the different domain
controllers based on domain names. This is required because future commits will
avoid calling into libresolv's code which resolv_wrapper takes care of.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
62584f3d by Stefan Metzmacher at 2019-01-15T06:53:22Z
selftest: setup_dns_hub

Start the central dns forwarder on interface 64

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
5f02a6af by Volker Lendecke at 2019-01-15T06:53:22Z
selftest: Add dns_hub deps

All the DCs want the dns forwarder

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
99b77533 by Volker Lendecke at 2019-01-15T06:53:22Z
selftest: Use dns_hub's resolv.conf

Pass it as RESOLV_CONF envvar everywhere

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
9563fcf6 by Volker Lendecke at 2019-01-15T06:53:22Z
samba_dnsupdate: With dns_hub, we don't need resolv_wrap

Best viewed with git show -b

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
4b4ae005 by Volker Lendecke at 2019-01-15T06:53:22Z
addns: Async ads_dns_lookup_srv

Use dns_lookup_send/recv to get SRV records. This avoids synchronous libresolv
calls and provides the infrastructure to get dsgetdcname async.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3f4bd61b by Karolin Seeger at 2019-01-15T09:59:52Z
WHATSNEW: Add release note for Samba 4.10.0rc1.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
9537e5d2 by Karolin Seeger at 2019-01-15T10:01:07Z
VERSION: Bump version up to 4.10.0rc1...

and disable GIT_SNAPSHOT for the release.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
75106e05 by Karolin Seeger at 2019-01-15T10:02:10Z
VERSION: Bump version up to 4.10.0rc2...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
f2d8308c by Volker Lendecke at 2019-01-15T10:16:00Z
addns: Async ads_dns_lookup_ns

Use dns_lookup_send/recv to get NS records

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Tue Jan 15 11:16:00 CET 2019 on sn-devel-144

- - - - -
118b492d by Karolin Seeger at 2019-01-15T10:19:40Z
Revert "VERSION: Bump version up to 4.10.0rc2..."

This reverts commit 75106e05b42e5d4629aacfa941213745d9d6e819.

- - - - -
a4e37680 by Karolin Seeger at 2019-01-15T10:19:43Z
Revert "VERSION: Bump version up to 4.10.0rc1..."

This reverts commit 9537e5d2a14a35bcba659d679d2d156de6c678cc.

- - - - -
08f27a00 by Karolin Seeger at 2019-01-15T10:19:46Z
Revert "WHATSNEW: Add release note for Samba 4.10.0rc1."

This reverts commit 3f4bd61b8408f8d97817023f07de20746ce54f90.

- - - - -
33de2f07 by Stefan Metzmacher at 2019-01-15T10:23:20Z
Merge remote-tracking branch 'origin/v4-10-test' into HEAD

This is a noop just to get the history of origin/v4-10-test
inline with master before the real 4.10.0rc1.

Signed-off-by: Stefan Metzmacher <metze at samba.org>

- - - - -
b9156260 by Stefan Metzmacher at 2019-01-15T10:28:13Z
talloc: version 2.1.15

* Deprecate talloc_set_memlimit() and talloc_autofree_context()
* Fix undefined behavior in talloc_memdup
* The build uses python3 by default:
* --extra-python would take python2 now
* To build with python2 only use:
  PYTHON=python2 ./configure
  PYTHON=python2 make
  PYTHON=python2 make install

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Karolin Seeger <kseeger at samba.org>

- - - - -
1c73f386 by Stefan Metzmacher at 2019-01-15T10:29:24Z
tevent: version 0.9.38

* Deprecate tevent wrapper api again
* Build fixes
* The build uses python3 by default:
* --extra-python would take python2 now
* To build with python2 only use:
  PYTHON=python2 ./configure
  PYTHON=python2 make
  PYTHON=python2 make install

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Karolin Seeger <kseeger at samba.org>

- - - - -
340cb9ca by Stefan Metzmacher at 2019-01-15T10:29:56Z
ldb: version 1.5.2

* Build fixes
* dirsync: Allow arbitrary length cookies
  (bug #13686)
* The build uses python3 by default:
* --extra-python would take python2 now
* To build with python2 only use:
  PYTHON=python2 ./configure
  PYTHON=python2 make
  PYTHON=python2 make install

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Karolin Seeger <kseeger at samba.org>

- - - - -
86253ce3 by Karolin Seeger at 2019-01-15T10:31:42Z
WHATSNEW: Add release note for Samba 4.10.0rc1.

Signed-off-by: Karolin Seeger <kseeger at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

(replaces commit 3f4bd61b8408f8d97817023f07de20746ce54f90)

- - - - -
1c9e1bcd by Karolin Seeger at 2019-01-15T10:33:53Z
VERSION: Bump version up to 4.10.0rc1...

and disable GIT_SNAPSHOT for the release.

Signed-off-by: Karolin Seeger <kseeger at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

(replaces commit 9537e5d2a14a35bcba659d679d2d156de6c678cc)

- - - - -
248c234e by Karolin Seeger at 2019-01-15T10:34:39Z
VERSION: Bump version up to 4.10.0rc2...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

(replaces commit 75106e05b42e5d4629aacfa941213745d9d6e819)

- - - - -
529b2920 by Justin Stephenson at 2019-02-04T08:50:08Z
s3:libsmb: Honor disable_netbios option in smbsock_connect_send

If disable_netbios is set, return before the tevent timer is triggered
to prevent outgoing netbios connections.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13727

Signed-off-by: Justin Stephenson <jstephen at redhat.com>
Reviewed-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit c324f84a2fa25e29d2f7879fbcd35ce0e76a78f8)

- - - - -
f43abe39 by Tim Beale at 2019-02-04T08:50:08Z
python/gpclass: Convert gpclass to use s3 SMB Python bindings

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 3b2e86bba1f6b22b144b07db454b842f0c0779ae)

- - - - -
3ed03bd2 by Tim Beale at 2019-02-04T08:50:08Z
s3:pylibsmb: Add .set_acl API to SMB py bindings

This is pretty similar code to py_smb_getacl(), except it's calling
cli_set_security_descriptor() instead of cli_query_security_descriptor()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit b982811b60521624f1f600841ffa05e306eb936a)

- - - - -
b47e42e0 by Tim Beale at 2019-02-04T08:50:08Z
netcmd: Change SMB flags from s4 Py bindings to s3

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 345746ea5f4cca2620421ac0c8a649a596f5c19d)

- - - - -
c9fdea2e by Tim Beale at 2019-02-04T08:50:08Z
s3:pylibsmb: Add FILE_READ_ATTRIBUTES access to .loadfile() API

Add FILE_READ_ATTRIBUTES when opening the file handle, as we need to
read the file's size.

The .loadfile() API can end up calling cli_qfileinfo_basic() to get the
file size. This can end up doing a 'FILE_ALL_INFORMATION' SMBv2 request
underneath, which the MS-SMB2 spec (section 3.3.5.20.1 Handling
SMB2_0_INFO_FILE) says the file handle must have FILE_READ_ATTRIBUTES
access granted.

I noticed this problem when running .loadfile() against the NTVFS
server.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 0304b08de5ba1b4a6e08568a559c52f7d9e943d3)

- - - - -
c4323c00 by Tim Beale at 2019-02-04T08:50:08Z
netcmd: Change GPO commands to use s3 SMB Py bindings

This means we can now use GPO commands on a DC that has SMBv1 disabled.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 08f1627cb3fa1c9ff7a0f74e32874d305647dc42)

- - - - -
9d9b0056 by Tim Beale at 2019-02-04T08:50:08Z
s4:pysmb: Add error log that the s4 bindings are deprecated

We plan to delete the s4 SMB Python bindings in the next Samba release
after v4.10, but first give external consumers a heads-up, just in case
they are currently using the s4 bindings.

Note the auth_log tests still use the s4 bindings, but all user-facing
tools should now be updated to use the s3 bindings.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 6a29e63f32c0024587020fc1f92b3d1ecaa0afbc)

- - - - -
6158ca6e by Tim Beale at 2019-02-04T08:50:08Z
tests: Run samba_tool.gpo tests against backup testenvs

Run the GPO tests against the backup/restore testenvs.

Because the backup/restore preserves the NTACLs of the sysvol files,
running the GPO tests against the backup testenvs is a good sanity-
check. If fact it highlights that there is currently a problem with
restoring the GPO files - this shows up in 'samba-tool gpo aclcheck',
but we never noticed it until now.

NTACL backup works slightly different for offline backups, and rename
backups end up with more sysvol files, so run the tests against both
these envs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 2a7372da7b84921b247cefc23d7003b8056d74a4)

- - - - -
e682347b by Tim Beale at 2019-02-04T08:50:09Z
ntacls: Pass correct use_ntvfs through to setntacl()

We were already checking the smb.conf to see if it uses the NTVFS file
server or the default smbd server. However, we weren't passing this
through to the setntacl() call.

This fixes the problem we noticed with 'samba-tool gpo aclcheck' failing
after a restore.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit f3fe96fc2e2d942b4a2e241777b5ee12b2295b97)

- - - - -
349cfec0 by Tim Beale at 2019-02-04T08:50:09Z
tests: Run GPO commands against testenv with SMBv1 disabled

Just to prove that they work across SMBv2.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit da33c2c4e4849f0985b08fbdc58cbd59b8426ec6)

- - - - -
7399fe07 by Tim Beale at 2019-02-04T08:50:09Z
selftest: Give the backup testenvs a 'test1' share

The ntacls_backup tests use the test1 share, and we want to run them
against the restoredc (which has SMBv1 disabled).

The xattr.tdb file is needed for the backend_obj.wrap_getxattr() call
(in ntacls.py) to work.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit a310de2db13c02a602e74139cb47ea9a25628e01)

- - - - -
f088f070 by Tim Beale at 2019-02-04T08:50:09Z
tests: Run ntacls_backup tests against testenv with SMBv1 disabled

Just to prove that the NTACL backup works over SMBv2.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 7fb93eaca74ffe17bbe7255210dd3090afe8d5dc)

- - - - -
26cd687f by Tim Beale at 2019-02-04T08:50:09Z
netcmd: Try to improve domain backup error message

I ran this command as non-root by mistake and didn't find the error
message particularly helpful. Tweak the error message so it reminds the
user that they should be root. Also display the path we're looking for
the sam.ldb file in, to give them more clues.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Mon Jan 21 16:34:06 CET 2019 on sn-devel-144

(cherry picked from commit 10e54a095f005c0988a7e5e8a35cea6200197854)

- - - - -
16b1971e by Tim Beale at 2019-02-04T08:50:09Z
WHATSNEW: Update for Bug 13676 changes in Samba 4.10

Although it's unlikely that users will be using the 'smb' Python
bindings, it's probably worth noting in the release notes that these
bindings will be deprecated in future releases.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13676

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
a4466ec4 by Volker Lendecke at 2019-02-04T12:09:49Z
ctdb: Print locks latency in machinereadable stats

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13742
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Wed Jan 16 05:34:17 CET 2019 on sn-devel-144

(cherry picked from commit 193a0d6f01372604b925d1972591062a0bb2400f)

Autobuild-User(v4-10-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-10-test): Mon Feb  4 13:09:49 CET 2019 on sn-devel-144

- - - - -
9859cc5c by Anoop C S at 2019-02-05T19:41:24Z
WHATSNEW: Add missing parenthesis

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13757

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
94aff506 by Günther Deschner at 2019-02-05T19:41:24Z
WHATSNEW: fix typo.

Guenther

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13757

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Noel Power <npower at samba.org>

- - - - -
9ae7ffa5 by Anoop C S at 2019-02-05T19:41:24Z
vfs_glusterfs: Adapt to changes in libgfapi signatures

VFS module for GlusterFS fails to compile due to recent changes done to
some API signatures. Therefore adding missing arguments to those APIs
adapting to new signatures.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13330

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sun Feb  3 17:00:33 CET 2019 on sn-devel-144

(cherry picked from commit 0e3eda5bab5ae9316a42725aea048fb350020ec7)

- - - - -
334f1e6f by Tim Beale at 2019-02-05T19:41:24Z
join: Fix TypeError when handling exception

When we can't resolve a domain name, we were inadvertently throwing a
TypeError whilst trying to output a helpful message. E.g.

ERROR(<class 'TypeError'>): uncaught exception - 'NTSTATUSError' object
does not support indexing

Instead of indexing the object, we want to index the Exception.args so
that we just display the string portion of the exception error.

The same problem is also present for the domain trust commands.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13747

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Rowland Penny <rpenny at samba.org>
Reviewed-by: Jeremy Allison <rpenny at samba.org>
(cherry picked from commit 3bb7808984c163a7bba66fb983411d1281589722)

- - - - -
56549fd0 by Tim Beale at 2019-02-05T19:41:24Z
join: Throw CommandError instead of Exception for simple errors

Throwing an exception here still dumps out the Python stack trace, which
can be a little disconcerting for users.

In this case, the stack trace isn't going to really help at all (the
problem is pretty obvious), and it obscures the useful message
explaining what went wrong.

Throw a CommandError instead, which samba-tool will catch and display
more nicely.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13747

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Rowland Penny <rpenny at samba.org>
Reviewed-by: Jeremy Allison <rpenny at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Jan 16 22:11:04 CET 2019 on sn-devel-144

(cherry picked from commit 9e4b08f4c384b8cae5ad853a7be7cf03e2749be5)

- - - - -
6990f501 by Jeremy Allison at 2019-02-05T19:41:24Z
python: dns_hub: Fix indentation of 'raise' on error.

Remove second socket.sendto().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13750
Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
(cherry picked from commit dd231a16a338cfe63d12cadf0fc3e2ebe7cac828)

- - - - -
520c062d by Jeremy Allison at 2019-02-05T19:41:24Z
libcli: dns: Change internal DNS_REQUEST_TIMEOUT from 2 to 10 seconds.

Should make us more robust when dealing with slow DNS servers.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13750

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Tue Jan 22 23:37:16 CET 2019 on sn-devel-144

(cherry picked from commit 36c42e6d629a0d4b0e5c60bdd68e08a8e60c477a)

- - - - -
9d2e05b1 by Stefan Metzmacher at 2019-02-05T19:41:24Z
manpages/samba.7.xml: smbcontrol can also work with 'samba'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Björn Baumbach <bbaumbach at samba.org>
(cherry picked from commit 12b9adec3ff48f4356f9ff865891dc3c652ff86b)

- - - - -
f105c379 by Stefan Metzmacher at 2019-02-05T19:41:25Z
s4:messaging: add support 'smbcontrol <pid> debug/debuglevel'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Björn Baumbach <bbaumbach at samba.org>
(cherry picked from commit 3a0c1da432c53de234b54bac90a3fb84534994eb)

- - - - -
45c19f4c by Stefan Metzmacher at 2019-02-05T19:41:25Z
s4:server: avoid using pid=0 for the parent 'samba' process

It confuses the 'samba-tool processes' output and log messages.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Björn Baumbach <bbaumbach at samba.org>
(cherry picked from commit 5bd7a8e5685caa09067745b108ef7e53e3108e97)

- - - - -
902de86d by Stefan Metzmacher at 2019-02-05T19:41:25Z
s4:server: add support for 'smbcontrol samba shutdown'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Björn Baumbach <bbaumbach at samba.org>
(cherry picked from commit 832776c0fcf7cc658c128765514755c2d15b06a6)

- - - - -
a2e88974 by Stefan Metzmacher at 2019-02-05T19:41:25Z
selftest:Samba4: use 'smbcontrol samba shutdown'

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13752

Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Björn Baumbach <bbaumbach at samba.org>

Autobuild-User(master): Andreas Schneider <asn at cryptomilk.org>
Autobuild-Date(master): Wed Jan 30 01:51:48 CET 2019 on sn-devel-144

(cherry picked from commit d03991f569b54ae0a11911b622107fbae701715d)

- - - - -
f8773e8a by Günther Deschner at 2019-02-05T19:41:25Z
s3-vfs: add glusterfs_fuse vfs module.

This module only implements the get_real_filename function by accessing
a distinct extended attribute that is available over a glusterfs fuse
mount.

By implementing this vfs function users of a glusterfs fuse mount
achieve a much better performance in create based workloads where samba
then can avoid trying multiple case folding options to detect the real
filename.

Patch is based on an initial patch provided by
Poornima G <pgurusid at redhat.com>

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13774

Guenther

Signed-off-by: Günther Deschner <gd at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Günther Deschner <gd at samba.org>
Autobuild-Date(master): Tue Jan 22 18:37:56 CET 2019 on sn-devel-144

(cherry picked from commit adffe0dcf002aa4721dc7897261895e3486d5271)

- - - - -
676b5493 by Anoop C S at 2019-02-05T22:59:06Z
s3-vfs: Use ENOATTR in errno comparison for getxattr

* ENODATA is not defined in FreeBSD
* ENOATTR is defined to be a synonym for ENODATA in Linux
* In its absence Samba already defines ENOATTR to either
  ENODATA or ENOENT

Thus it is safe and correct to compare with ENOATTR rather
than ENODATA.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13774

Signed-off-by: Anoop C S <anoopcs at redhat.com>
Reviewed-by: Uri Simchoni <uri at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Jan 23 21:59:10 CET 2019 on sn-devel-144

(cherry picked from commit c99402724a65f4e1f8ed4dcd236a43e0603bef0a)

Autobuild-User(v4-10-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-10-test): Tue Feb  5 23:59:06 CET 2019 on sn-devel-144

- - - - -
d902eec5 by Karolin Seeger at 2019-02-06T07:57:04Z
WHATSNEW: Add release notes for Samba 4.10.0rc2.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
61e65482 by Karolin Seeger at 2019-02-06T07:59:54Z
VERSION: Diable GIT_SNAPSHOT for the 4.10.0rc2 release.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
0c75bfe6 by Karolin Seeger at 2019-02-06T08:00:43Z
VERSION: Bump version up to 4.10.0rc2...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
7fc34817 by Tim Beale at 2019-02-13T10:41:10Z
ldb: Avoid inefficient one-level searches

Commit 88ae60ed186c9 introduced a problem that made one-level
searches inefficient if there were a lot of child objects in the same
level, and the requested object didn't exist. Basically, it ignored the
case where ldb_kv_index_dn() returned LDB_ERR_NO_SUCH_OBJECT, i.e. the
indexed lookup was successful, but didn't find a match. At which point,
there was no more processing we needed to do.

The behaviour after 88ae60ed186c9 was to fall-through and run the
ldb_kv_index_filter() function over *all* the children. This still
returned the correct result, but could be costly if there were a lot of
children.

The case 88ae60ed186c9 was trying to fix was where we could not do
an indexed search (e.g. trying to match on a 'attribute=*' filter). In
which case we want to ignore the LDB_ERR_OPERATIONS_ERROR and just run
ldb_kv_index_filter() over all the children. This is still more
efficient than the fallback of doing a full database scan.

This patch adds in a short-circuit for the NO_SUCH_OBJECT case, so we
can skip the unnecessary ldb_kv_index_filter() work.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 9a893f9613bd6440abd8e487d22a39ab5b82a7b9)

- - - - -
f7774530 by Tim Beale at 2019-02-13T10:41:10Z
ldb: Remove comment that no longer makes sense

This comment was written before the GUID_index_attribute block of code
existed. So we now *do* load the index values and *do* check for a
strict intersect, so the comment is redundant.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 72928444823c5b18ac9ef98e7432c999d70aa571)

- - - - -
62fea7e9 by Tim Beale at 2019-02-13T10:41:10Z
ldb: Elaborate on ldb_kv_search_indexed() comments

Disclaimer: this is based on my limited understanding of what the code
is doing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 132600685b8c5d4964f20634cd7a64b14f41cfa7)

- - - - -
2a915942 by Tim Beale at 2019-02-13T10:41:10Z
ldb: Rename variable

The old name confused me because it's not really related to the
one-level index at all. It's the result from evaluating the indexed
search specified in the ac->tree.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 57a565b2fd680fc1a34f4ab91c6f6314f68ef67f)

- - - - -
bb850a07 by Andrew Bartlett at 2019-02-13T10:41:11Z
ldb: Add even more comments on what strict does to the list intersections

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
(cherry picked from commit e7f524fd2128aacb82e980652af8eb6fd275e1a8)

- - - - -
e21e24d8 by Andrew Bartlett at 2019-02-13T15:24:32Z
ldb: Release ldb 1.5.3

* Avoid inefficient one-level searches (bug 13762)
* The test api.py should not rely on order of entries in dict (bug 13772)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13772

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
(cherry picked from commit 5e716c0256a6bec92e7855ccfc077a328320f2ea)

Autobuild-User(v4-10-test): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(v4-10-test): Wed Feb 13 16:24:32 CET 2019 on sn-devel-144

- - - - -
7621c622 by Günther Deschner at 2019-02-21T11:30:41Z
s3-smbd: use fruit:model string for mDNS registration

With this change we now allow to modify the icon to represent Samba in
Finder. Possible values are at least:

fruit:model = iMac
fruit:model = MacBook
fruit:model = MacPro
fruit:model = Xserve
fruit:model = RackMac

Prior to this change we only displayed the correct icon when a mac
client negotiated the apple create context over SMB.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13746

Based on proposed patch from Rouven WEILER <Rouven_Weiler at gmx.net>

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Günther Deschner <gd at samba.org>
Autobuild-Date(master): Tue Jan 15 21:27:20 CET 2019 on sn-devel-144

(cherry picked from commit 538ce72f1b2fa78450e3b711e58bd0e238faf466)

- - - - -
3d0450ef by David Disseldorp at 2019-02-21T11:30:41Z
printing: drop pcap_cache_loaded() guard around load_printers()

Add the pcap_cache_loaded() check to load_printers() and return early
if it returns false. This simplifies callers in preparation for checking
lp_load_printers() in the printcap cache update code-path.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13766

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Noel Power <npower at samba.org>
(cherry picked from commit 0ae7c3144a30910adb1e54cf46d54d42a1036839)

- - - - -
d6dbd33f by David Disseldorp at 2019-02-21T11:30:42Z
printing: check lp_load_printers() prior to pcap cache update

Avoid explicit and housekeeping timer triggered printcap cache updates
if lp_load_printers() is disabled.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13766

Signed-off-by: David Disseldorp <ddiss at samba.org>

Autobuild-User(master): Noel Power <npower at samba.org>
Autobuild-Date(master): Fri Feb  1 19:25:03 CET 2019 on sn-devel-144

(cherry picked from commit 6a77237c50dd258521f356af0b5dc9942dd5592e)

- - - - -
f68aab42 by Jeremy Allison at 2019-02-21T11:30:42Z
s3: tests: Add regression test for smbd crash on share force group change with existing connection.

Mark as known fail for now.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13690

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>
(cherry picked from commit 7b21b4c1f538650f23ec77fb3c02fe1e224d89aa)

- - - - -
eba5d055 by Jeremy Allison at 2019-02-21T11:30:42Z
smbd: uid: Don't crash if 'force group' is added to an existing share connection.

smbd could crash if "force group" is added to a
share definition whilst an existing connection
to that share exists. In that case, don't change
the existing credentials for force group, only
do so for new connections.

Remove knownfail from regression test.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13690

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Fri Jan 25 16:31:27 CET 2019 on sn-devel-144

(cherry picked from commit e37f9956c1f2416408bad048a4618f6366086b6a)

- - - - -
72bcae77 by Jeremy Allison at 2019-02-21T11:30:42Z
s3: VFS: vfs_fruit. Fix the NetAtalk deny mode compatibility code.

This exhibited itself as a problem with OFD locks reported
as:

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13770

However, due to underlying bugs in the vfs_fruit
code the file locks were not being properly applied.

There are two problems in fruit_check_access().

Problem #1:

Inside fruit_check_access() we have:

flags = fcntl(fsp->fh->fd, F_GETFL);
..
if (flags & (O_RDONLY|O_RDWR)) {

We shouldn't be calling fcntl(fsp->fh->fd, ..) directly.
fsp->fh->fd may be a made up number from an underlying
VFS module that has no meaning to a system call.

Secondly, in all POSIX systems - O_RDONLY is defined as
*zero*. O_RDWR = 2.

Which means flags & (O_RDONLY|O_RDWR) becomes (flags & 2),
not what we actually thought.

Problem #2:

deny_mode is *not* a bitmask, it's a set of discrete values.

Inside fruit_check_access() we have:

if (deny_mode & DENY_READ) and also (deny_mode & DENY_WRITE)

However, deny modes are defined as:

/* deny modes */
define DENY_DOS 0
define DENY_ALL 1
define DENY_WRITE 2
define DENY_READ 3
define DENY_NONE 4
define DENY_FCB 7

so if deny_mode = DENY_WRITE, or if deny_mode = DENY_READ
then it's going to trigger both the if (deny_mode & DENY_READ)
*and* the (deny_mode & DENY_WRITE) conditions.

These problems allowed the original test test_netatalk_lock code to
pass (which was added for BUG: https://bugzilla.samba.org/show_bug.cgi?id=13584
to demonstrate the lock order violation).

This patch refactors the fruit_check_access()
code to be much simpler (IMHO) to understand.

Firstly, pass in the SMB1/2 share mode, not old
DOS deny modes.

Secondly, read all the possible NetAtalk locks
into local variables:

netatalk_already_open_for_reading
netatalk_already_open_with_deny_read
netatalk_already_open_for_writing
netatalk_already_open_with_deny_write

Then do the share mode/access mode checks
with the requested values against any stored
netatalk modes/access modes.

Finally add in NetATalk compatible locks
that represent our share modes/access modes
into the file, with an early return if we don't
have FILE_READ_DATA (in which case we can't
write locks anyway).

The patch is easier to understand by looking
at the completed patched fruit_check_access()
function, rather than trying to look at the
diff.

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Böhme <slow at samba.org>
(cherry picked from commit 3204dc66f6801a7c8c87c48f601e0ebdee9e3d40)

- - - - -
fbc8ea71 by Jeremy Allison at 2019-02-21T11:30:43Z
s4: torture: vfs_fruit. Change test_fruit_locking_conflict() to match the vfs_fruit working server code.

Originally added for BUG: https://bugzilla.samba.org/show_bug.cgi?id=13584
to demonstrate a lock order violation, this test
exposed problems in the mapping of SMB1/2 share modes
and open modes to NetATalk modes once we moved to OFD locks.

Change the test slightly (and add comments)
so it demonstrates working NetATalk share modes
on an open file.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13770

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Böhme <slow at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Fri Feb  8 23:26:46 CET 2019 on sn-devel-144

(cherry picked from commit 28990e4ba23695ecf264117efad90cc4e573302e)

- - - - -
fcd5865f by Andreas Schneider at 2019-02-21T11:30:43Z
s3:vfs: Initialize pid to 0 in test_netatalk_lock()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13770

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 2ff2594b2bd878928cec30bc72a95a6d38bee154)

- - - - -
1ed75df5 by Andreas Schneider at 2019-02-21T11:30:43Z
s3:vfs: Correctly check if OFD locks should be enabled or not

Also the smb.conf options should only be checked once and a reload of
the config should not switch to a different locking mode.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13770

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Sat Feb  9 03:43:50 CET 2019 on sn-devel-144

(cherry picked from commit 7ff94b18e2e39567ef7a208084cc5c914c39d3bd)

- - - - -
9a530ee4 by Ralph Boehme at 2019-02-21T11:30:44Z
tldap: avoid a use after free crash

I saw the following crash in tldap in the winbindd idmap child on a
member server after messing with the LDAP server on the DC:

0  0x00007f77ea9a307a in __GI___waitpid (pid=9815, stat_loc=stat_loc at entry=0x7ffe77569eb0, options=options at entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
1  0x00007f77ea91bfbb in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:148
2  0x00007f77edd8c24b in smb_panic_s3 (why=0x7f77f08e6e88 "Bad talloc magic value - access after free") at ../source3/lib/util.c:828
3  0x00007f77f15afe85 in smb_panic (why=0x7f77f08e6e88 "Bad talloc magic value - access after free") at ../lib/util/fault.c:170
4  0x00007f77f08e2678 in talloc_abort (reason=0x7f77f08e6e88 "Bad talloc magic value - access after free") at ../lib/talloc/talloc.c:472
5  0x00007f77f08e268b in talloc_abort_access_after_free () at ../lib/talloc/talloc.c:477
6  0x00007f77f08e2710 in talloc_chunk_from_ptr (ptr=0x55da7605a020) at ../lib/talloc/talloc.c:494
7  0x00007f77f08e4a19 in _talloc_free (ptr=0x55da7605a020, location=0x7f77e181474d "../source3/lib/tldap.c:1918") at ../lib/talloc/talloc.c:1716
8  0x00007f77e180b65c in tldap_search_all_done (subreq=0x55da7605a020) at ../source3/lib/tldap.c:1918
9  0x00007f77f0af0fd0 in _tevent_req_notify_callback (req=0x55da7605a020, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:125
10 0x00007f77f0af10a5 in tevent_req_finish (req=0x55da7605a020, state=TEVENT_REQ_USER_ERROR, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:162
11 0x00007f77f0af1113 in _tevent_req_error (req=0x55da7605a020, error=9780923860630110289, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:180
12 0x00007f77e180781a in tevent_req_ldap_error (req=0x55da7605a020, rc=...) at ../source3/lib/tldap.c:47
13 0x00007f77e180b2c4 in tldap_search_done (subreq=0x55da76058280) at ../source3/lib/tldap.c:1813
14 0x00007f77f0af0fd0 in _tevent_req_notify_callback (req=0x55da76058280, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:125
15 0x00007f77f0af10a5 in tevent_req_finish (req=0x55da76058280, state=TEVENT_REQ_USER_ERROR, location=0x7f77e1813e50 "../source3/lib/tldap.c:47") at ../lib/tevent/tevent_req.c:162
16 0x00007f77f0af11cd in tevent_req_trigger (ev=0x55da760526c0, im=0x55da76058360, private_data=0x55da76058280) at ../lib/tevent/tevent_req.c:219
17 0x00007f77f0af0378 in tevent_common_loop_immediate (ev=0x55da760526c0) at ../lib/tevent/tevent_immediate.c:135
18 0x00007f77f0af8b8f in epoll_event_loop_once (ev=0x55da760526c0, location=0x7f77f0af92b0 "../lib/tevent/tevent_req.c:269") at ../lib/tevent/tevent_epoll.c:911
19 0x00007f77f0af5925 in std_event_loop_once (ev=0x55da760526c0, location=0x7f77f0af92b0 "../lib/tevent/tevent_req.c:269") at ../lib/tevent/tevent_standard.c:114
20 0x00007f77f0aef201 in _tevent_loop_once (ev=0x55da760526c0, location=0x7f77f0af92b0 "../lib/tevent/tevent_req.c:269") at ../lib/tevent/tevent.c:725
21 0x00007f77f0af1361 in tevent_req_poll (req=0x55da7605eed0, ev=0x55da760526c0) at ../lib/tevent/tevent_req.c:269
22 0x00007f77e180fec9 in tldap_gensec_bind (ctx=0x55da76051ec0, creds=0x55da76052250, target_service=0x7f77e18164b3 "ldap", target_hostname=0x55da7605d182 "dc1.sdom1.site", target_principal=0x0, lp_ctx=0x55da76052180, gensec_features=6) at ../source3/lib/tldap_gensec_bind.c:358
23 0x00007f77e1810d21 in idmap_ad_get_tldap_ctx (mem_ctx=0x55da76050510, domname=0x55da76051d50 "sdom1", pld=0x55da76050518) at ../source3/winbindd/idmap_ad.c:326
24 0x00007f77e1811056 in idmap_ad_context_create (mem_ctx=0x55da76059c00, dom=0x55da76059c00, domname=0x55da76051d50 "sdom1", pctx=0x7ffe7756a5f8) at ../source3/winbindd/idmap_ad.c:374
25 0x00007f77e18119c0 in idmap_ad_get_context (dom=0x55da76059c00, pctx=0x7ffe7756a640) at ../source3/winbindd/idmap_ad.c:554
26 0x00007f77e181275b in idmap_ad_sids_to_unixids (dom=0x55da76059c00, ids=0x55da760518a0) at ../source3/winbindd/idmap_ad.c:784
27 0x00007f77e1813217 in idmap_ad_sids_to_unixids_retry (dom=0x55da76059c00, ids=0x55da760518a0) at ../source3/winbindd/idmap_ad.c:947
28 0x000055da7459ce05 in _wbint_Sids2UnixIDs (p=0x7ffe7756a870, r=0x55da76050860) at ../source3/winbindd/winbindd_dual_srv.c:202
29 0x000055da7460aa5e in api_wbint_Sids2UnixIDs (p=0x7ffe7756a870) at default/librpc/gen_ndr/srv_winbind.c:391
30 0x000055da7459c7f4 in winbindd_dual_ndrcmd (domain=0x0, state=0x7ffe7756abb8) at ../source3/winbindd/winbindd_dual_ndr.c:369
31 0x000055da7459828c in child_process_request (child=0x55da74874bc0 <static_idmap_child>, state=0x7ffe7756abb8) at ../source3/winbindd/winbindd_dual.c:666
32 0x000055da7459ae58 in child_handler (ev=0x55da7602c2b0, fde=0x55da7603f8a0, flags=1, private_data=0x7ffe7756abb0) at ../source3/winbindd/winbindd_dual.c:1567
33 0x00007f77f0af85f1 in epoll_event_loop (epoll_ev=0x55da76048b00, tvalp=0x7ffe7756aab0) at ../lib/tevent/tevent_epoll.c:728
34 0x00007f77f0af8c29 in epoll_event_loop_once (ev=0x55da7602c2b0, location=0x55da74628b08 "../source3/winbindd/winbindd_dual.c:1766") at ../lib/tevent/tevent_epoll.c:930
35 0x00007f77f0af5925 in std_event_loop_once (ev=0x55da7602c2b0, location=0x55da74628b08 "../source3/winbindd/winbindd_dual.c:1766") at ../lib/tevent/tevent_standard.c:114
36 0x00007f77f0aef201 in _tevent_loop_once (ev=0x55da7602c2b0, location=0x55da74628b08 "../source3/winbindd/winbindd_dual.c:1766") at ../lib/tevent/tevent.c:725
37 0x000055da7459b9e9 in fork_domain_child (child=0x55da74874bc0 <static_idmap_child>) at ../source3/winbindd/winbindd_dual.c:1766
38 0x000055da74596e96 in wb_child_request_waited (subreq=0x0) at ../source3/winbindd/winbindd_dual.c:188
39 0x00007f77f0af0fd0 in _tevent_req_notify_callback (req=0x55da7604f820, location=0x7f77f0af90f8 "../lib/tevent/tevent_queue.c:355") at ../lib/tevent/tevent_req.c:125
40 0x00007f77f0af10a5 in tevent_req_finish (req=0x55da7604f820, state=TEVENT_REQ_DONE, location=0x7f77f0af90f8 "../lib/tevent/tevent_queue.c:355") at ../lib/tevent/tevent_req.c:162
41 0x00007f77f0af10cd in _tevent_req_done (req=0x55da7604f820, location=0x7f77f0af90f8 "../lib/tevent/tevent_queue.c:355") at ../lib/tevent/tevent_req.c:168
42 0x00007f77f0af0cc1 in tevent_queue_wait_trigger (req=0x55da7604f820, private_data=0x0) at ../lib/tevent/tevent_queue.c:355
43 0x00007f77f0af06f2 in tevent_queue_immediate_trigger (ev=0x55da7602c2b0, im=0x55da760466a0, private_data=0x55da76046580) at ../lib/tevent/tevent_queue.c:149
44 0x00007f77f0af0378 in tevent_common_loop_immediate (ev=0x55da7602c2b0) at ../lib/tevent/tevent_immediate.c:135
45 0x00007f77f0af8b8f in epoll_event_loop_once (ev=0x55da7602c2b0, location=0x55da74612630 "../source3/winbindd/winbindd.c:1803") at ../lib/tevent/tevent_epoll.c:911
46 0x00007f77f0af5925 in std_event_loop_once (ev=0x55da7602c2b0, location=0x55da74612630 "../source3/winbindd/winbindd.c:1803") at ../lib/tevent/tevent_standard.c:114
47 0x00007f77f0aef201 in _tevent_loop_once (ev=0x55da7602c2b0, location=0x55da74612630 "../source3/winbindd/winbindd.c:1803") at ../lib/tevent/tevent.c:725
48 0x000055da74561431 in main (argc=2, argv=0x7ffe7756c968) at ../source3/winbindd/winbindd.c:1803

subreq is a child of the state of req which will already be free by the
callback of req.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13776

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit 9465292d4109f710f8fcd141a076f5c8278577bc)

- - - - -
6d2af886 by Ralph Boehme at 2019-02-21T11:30:44Z
tldap: avoid more use after free errors

See the previous commit for an explanation. :)

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13776

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Wed Feb  6 10:19:12 CET 2019 on sn-devel-144

(cherry picked from commit bf91ee0a9727cc392583fe84ad069204be758515)

- - - - -
2ba1d9dd by Andreas Schneider at 2019-02-21T11:30:44Z
librpc:ndr: Implement ndr_zero_memory()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 2a7086f1ac4ad91fb7cd958e9386abb996794ed1)

- - - - -
f6a44c80 by Andreas Schneider at 2019-02-21T11:30:45Z
librpc:ndr: Add NDR_ZERO_STRUCT(P) macros

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 666802a3db3115ca09f3ffed58c8e4a8cabd65de)

- - - - -
66bf14d8 by Andreas Schneider at 2019-02-21T11:30:45Z
pidl: Use NDR_ZERO_STRUCT(P) macros

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 532ce0d20a8016c1270ea689de627da8aa4abfdd)

- - - - -
9bf434a9 by Andreas Schneider at 2019-02-21T11:30:45Z
lib:util: Move discard_const(_p) to own header for libndr.h

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 96df6878ed521b7e744d703abb32a585500d3b94)

- - - - -
be6f8d41 by Andreas Schneider at 2019-02-21T11:30:45Z
waf: Do not install internal header

We should not install header files without an public API:

- memory.h
- safe_strings.h
- talloc_stack.h

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13778

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 6d232f3f7c64f9b01439326e0e9b6d9df9a0bcbb)

- - - - -
d8911eaa by Tim Beale at 2019-02-21T11:30:46Z
WHATSNEW: Add note that python2 support will be dropped on v4.11

Add a warning to Samba users that v4.10 will be the last Samba release
with python2 support.

I've reworked the existing text describing the different python2 build
options for 4.10. Hopefully this makes it slightly clearer.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13785

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>

- - - - -
a3a9630a by Christof Schmitt at 2019-02-21T15:26:50Z
waf: Check for libnscd

The check was in the old autoconf, but not in waf. As the code is still
in source3/lib/util_nscd.c, add the check for libnscd to allow building
and using the code.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13787

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>

Autobuild-User(master): Günther Deschner <gd at samba.org>
Autobuild-Date(master): Wed Feb 13 17:58:33 CET 2019 on sn-devel-144

(cherry picked from commit 3a793497796395ffa3efda5807bdb1ca8e09e35b)

Autobuild-User(v4-10-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-10-test): Thu Feb 21 16:26:51 CET 2019 on sn-devel-144

- - - - -
1f749eb6 by Karolin Seeger at 2019-02-22T07:59:15Z
WHATSNEW: Add release notes for Samba 4.10.0rc3.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
3d91947b by Karolin Seeger at 2019-02-22T07:59:50Z
VERSION: Disable GIT_SNAPSHOT for the 4.10.0rc3 release.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
9dde00c3 by Karolin Seeger at 2019-02-22T08:00:37Z
VERSION: Bump version up to 4.10.0rc4...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
fb1d5988 by Noel Power at 2019-02-22T12:44:34Z
buildtools/wafsamba: Avoid decode when using python2

To avoid problematic type checking for 'str' types which fail
when result from str.decode is used.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13777

Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 244e2a02796b2ee85b9db01cbea7043a7448a110)

Autobuild-User(v4-10-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-10-test): Fri Feb 22 13:44:34 CET 2019 on sn-devel-144

- - - - -
ba5a93c8 by David Mulder at 2019-02-26T12:00:10Z
Search for location of waf script

When calling make from the ldb, talloc, tdb, and
tevent bundles, we need to first find the
location of the waf script. Currently the build
fails since it can't find waf.

Fixes regression caused by a660b7f.

Signed-off-by: David Mulder <dmulder at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 8ecd7f756f0fafe5fd79dc4deb445883450c7cca)

- - - - -
50be2c58 by Lukas Slebodnik at 2019-02-26T12:00:10Z
tdb: Fix compatibility of wscript with older python

Traceback (most recent call last):
  File "tdb-1.3.17/third_party/waf/waflib/Scripting.py", line 158, in waf_entry_point
    run_commands()
  File "tdb-1.3.17/third_party/waf/waflib/Scripting.py", line 251, in run_commands
    ctx = run_command(cmd_name)
  File "tdb-1.3.17/third_party/waf/waflib/Scripting.py", line 235, in run_command
    ctx.execute()
  File "tdb-1.3.17/third_party/waf/waflib/Context.py", line 204, in execute
    self.recurse([os.path.dirname(g_module.root_path)])
  File "tdb-1.3.17/third_party/waf/waflib/Context.py", line 286, in recurse
    user_function(self)
  File "tdb-1.3.17/wscript", line 225, in testonly
    cmd = "BINDIR={} {}".format(blddir, sh_test)
ValueError: zero length field name in format

Signed-off-by: Lukas Slebodnik <lslebodn at fedoraproject.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Sun Jan 20 03:49:59 CET 2019 on sn-devel-144

(cherry picked from commit 0628ca2a558871402f9a17cc7f7a0c69d857ae19)

- - - - -
b6bb285d by Andreas Schneider at 2019-02-26T12:00:10Z
lib:tdb: Use C99 initializer for PyGetSetDef in pytdb

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
(cherry picked from commit 651ee7f205f4a7e31c791f7bb235275816747463)

- - - - -
a5284f9c by Andreas Schneider at 2019-02-26T12:00:11Z
lib:tdb: Use C99 initializer for tdb_header

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
(cherry picked from commit a1ce666d68745a9bc65bd2709c581ed5105f656d)

- - - - -
0130b999 by Andreas Schneider at 2019-02-26T12:00:11Z
lib:tdb: Use C99 initializer for tdb_logging_context

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
(cherry picked from commit 47b57a2fc87664366099913305dd8d3783734a1a)

- - - - -
dd2ec6de by Stefan Metzmacher at 2019-02-26T12:00:11Z
tdb: version 1.3.18

* Fix build problems with older python versions.
* C99 build fixes.
* Fix standalone build of tdb.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit f0d26dd1816f35a00abf52b640f42547ffdfa01b)

- - - - -
7bc0d67e by Stefan Metzmacher at 2019-02-26T12:00:11Z
talloc: version 2.1.16

* Fix standalone build of talloc.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 3fe1551b5347934a20b9161a23e6a16220c3aeb6)

- - - - -
6b125f6c by Andreas Schneider at 2019-02-26T12:00:11Z
lib:tevent: Use correct C99 initializer for tevent_req

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
(cherry picked from commit 0da2e9c215f882ea4c331ac4f17738913beb7404)

- - - - -
f8686546 by Douglas Bagnall at 2019-02-26T12:00:11Z
py_tevent: add_timer takes float argument

We were already using it that way.

Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Noel Power <npower at samba.org>
(cherry picked from commit 8294e68a4175e6116b38869866c42a1b7ba55b6a)

- - - - -
0da2d830 by Stefan Metzmacher at 2019-02-26T12:00:11Z
tevent: version 0.9.39

* py_tevent: add_timer takes float argument
* C99 build fixes.
* Fix standalone build of tevent.

Signed-off-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit db58a50296041ca57675daee15caea8850f1d3f8)

- - - - -
7d0902c2 by Andreas Schneider at 2019-02-26T12:00:11Z
lib:ldb: Use C99 initializer for PyGetSetDef in pyldb

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
(cherry picked from commit 85a5dc56e34298a60d7ef96f4178ccff20d40c82)

- - - - -
d16b81cf by Andreas Schneider at 2019-02-26T12:00:11Z
lib:ldb: Use C99 initializer for tdb_logging_context

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
(cherry picked from commit 3cd5d4b39713559d2f88e21a24f8190fe9ce075c)

- - - - -
942822e7 by Andreas Schneider at 2019-02-26T12:00:11Z
lib:ldb: Use correct C99 initializer for 'struct tm'

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
(cherry picked from commit 146ed0894e640fe1791f75fa8f50167b6cdfc06e)

- - - - -
bfa9353c by Lukas Slebodnik at 2019-02-26T12:00:12Z
ldb: The test api.py should not rely on order of entries in dict

Test failed on s390x but there is a simple reproducer for any
architecture.

The built-in function repr returns the canonical string representation
of the object. We needn't care about order attributes in string
representation. Therefore test should pass for any order.

    for i in {1..30}; do
        PYTHONHASHSEED=random \
        python2 -c 'import ldb; msg = ldb.Message(); msg.dn = ldb.Dn(ldb.Ldb(), "dc=foo29"); msg["dc"] = b"foo"; print(repr(msg)) '
    done

  ======================================================================
  FAIL: test_repr (__main__.LdbMsgTests)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "tests/python/api.py", line 2322, in test_repr
      "Message({'dn': Dn('dc=foo29'), 'dc': MessageElement(['foo'])})")
  AssertionError: "Message({'dc': MessageElement(['foo']), 'dn': Dn('dc=foo29')})" != "Message({'dn': Dn('dc=foo29'), 'dc': MessageElement(['foo'])})"
  ----------------------------------------------------------------------
  Ran 1025 tests in 29.146s
  FAILED (failures=1)

Signed-off-by: Lukas Slebodnik <lslebodn at fedoraproject.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn at samba.org>
(cherry picked from commit 7a7a5ccf501f89c530970bde072509ed86d7bd89)

- - - - -
aa13a462 by Lukas Slebodnik at 2019-02-26T12:00:12Z
CVE-2019-3824 ldb: Out of bound read in ldb_wildcard_compare

There is valgrind error in few tests tests/test-generic.sh
 91 echo "Test wildcard match"
 92 $VALGRIND ldbadd $LDBDIR/tests/test-wildcard.ldif  || exit 1
 93 $VALGRIND ldbsearch '(cn=test*multi)'  || exit 1
 95 $VALGRIND ldbsearch '(cn=*test_multi)'  || exit 1
 97 $VALGRIND ldbsearch '(cn=test*multi*test*multi)'  || exit 1

e.g.
  ==3098== Memcheck, a memory error detector
  ==3098== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
  ==3098== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
  ==3098== Command: ./bin/ldbsearch (cn=test*multi)
  ==3098==
  ==3098== Invalid read of size 1
  ==3098==    at 0x483CEE7: memchr (vg_replace_strmem.c:890)
  ==3098==    by 0x49A9073: memmem (in /usr/lib64/libc-2.28.9000.so)
  ==3098==    by 0x485DFE9: ldb_wildcard_compare (ldb_match.c:313)
  ==3098==    by 0x485DFE9: ldb_match_substring (ldb_match.c:360)
  ==3098==    by 0x485DFE9: ldb_match_message (ldb_match.c:572)
  ==3098==    by 0x558F8FA: search_func (ldb_kv_search.c:549)
  ==3098==    by 0x48C78CA: ??? (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x48C7A60: tdb_traverse_read (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x557B7C4: ltdb_traverse_fn (ldb_tdb.c:274)
  ==3098==    by 0x558FBFA: ldb_kv_search_full (ldb_kv_search.c:594)
  ==3098==    by 0x558FBFA: ldb_kv_search (ldb_kv_search.c:854)
  ==3098==    by 0x558E497: ldb_kv_callback (ldb_kv.c:1713)
  ==3098==    by 0x48FCD58: tevent_common_invoke_timer_handler (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==    by 0x48FCEFD: tevent_common_loop_timer_delay (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==    by 0x48FE14A: ??? (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==  Address 0x4b4ab81 is 0 bytes after a block of size 129 alloc'd
  ==3098==    at 0x483880B: malloc (vg_replace_malloc.c:309)
  ==3098==    by 0x491048B: talloc_strndup (in /usr/lib64/libtalloc.so.2.1.15)
  ==3098==    by 0x48593CA: ldb_casefold_default (ldb_utf8.c:59)
  ==3098==    by 0x485F68D: ldb_handler_fold (attrib_handlers.c:64)
  ==3098==    by 0x485DB88: ldb_wildcard_compare (ldb_match.c:257)
  ==3098==    by 0x485DB88: ldb_match_substring (ldb_match.c:360)
  ==3098==    by 0x485DB88: ldb_match_message (ldb_match.c:572)
  ==3098==    by 0x558F8FA: search_func (ldb_kv_search.c:549)
  ==3098==    by 0x48C78CA: ??? (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x48C7A60: tdb_traverse_read (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x557B7C4: ltdb_traverse_fn (ldb_tdb.c:274)
  ==3098==    by 0x558FBFA: ldb_kv_search_full (ldb_kv_search.c:594)
  ==3098==    by 0x558FBFA: ldb_kv_search (ldb_kv_search.c:854)
  ==3098==    by 0x558E497: ldb_kv_callback (ldb_kv.c:1713)
  ==3098==    by 0x48FCD58: tevent_common_invoke_timer_handler (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==
  # record 1
  dn: cn=test_multi_test_multi_test_multi,o=University of Michigan,c=TEST
  cn: test_multi_test_multi_test_multi
  description: test multi wildcards matching
  objectclass: person
  sn: multi_test
  name: test_multi_test_multi_test_multi
  distinguishedName: cn=test_multi_test_multi_test_multi,o=University of Michiga
   n,c=TEST

  # returned 1 records
  # 1 entries
  # 0 referrals

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Lukas Slebodnik <lslebodn at fedoraproject.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
(cherry picked from commit 3674b0891afb016c83763520b87e9f190dcfe884)

- - - - -
e9afae48 by Andrew Bartlett at 2019-02-26T12:00:12Z
CVE-2019-3824 ldb: Extra comments to clarify no pointer wrap in wildcard processing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
(cherry picked from commit 745b99fc6b75db33cdb0a58df1a3f2a5063bc76e)

- - - - -
4cd0abe3 by Andrew Bartlett at 2019-02-26T12:00:12Z
CVE-2019-3824 ldb: Improve code style and layout in wildcard processing

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>
(cherry picked from commit 9427806f7298d71bd7edfbdda7506ec63f15dda1)

- - - - -
9a0ace32 by Gary Lockyer at 2019-02-26T12:00:12Z
CVE-2019-3824 ldb: ldb_parse_tree use talloc_zero

Initialise the created ldb_parse_tree with talloc_zero, this ensures
that it is correctly initialised if inadvertently passed to a function
expecting a different operation type.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 8d34d172092f71baad0d777567e49aebfa07313d)

- - - - -
41fd2cde by Gary Lockyer at 2019-02-26T12:00:12Z
CVE-2019-3824 ldb: wildcard_match check tree operation

Check the operation type of the passed parse tree, and return
LDB_INAPPROPRIATE_MATCH if the operation is not LDB_OP_SUBSTRING.

A query of "attribute=*" gets parsed as LDB_OP_PRESENT, checking the
operation and failing ldb_wildcard_match should help prevent confusion
writing tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 34383981a0c40860f71a4451ff8fd752e1b67666)

- - - - -
aecd14f8 by Gary Lockyer at 2019-02-26T12:00:12Z
CVE-2019-3824 ldb: wildcard_match end of data check

ldb_handler_copy and ldb_val_dup over allocate by one and add a trailing '\0'
to the data, to make them safe to use the C string functions on.

However testing for the trailing '\0' is not the correct way to test for
the end of a value, the length should be checked instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 42f0f57eb819ce6b68a8c5b3b53123b83ec917e3)

- - - - -
21a44989 by Gary Lockyer at 2019-02-26T12:00:12Z
CVE-2019-3824 ldb: Add tests for ldb_wildcard_match

Add cmocka tests for ldb_wildcard_match.

Running test_wildcard_match under valgrind reproduces
 CVE-2019-3824 out of bounds read in wildcard compare (bug 13773)

 valgrind --suppressions=lib/ldb/tests/ldb_match_test.valgrind\
          bin/ldb_match_test

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 45b75db50f5c1a7c8c38af59a62fccee5401c845)

- - - - -
97fcdfb5 by Stefan Metzmacher at 2019-02-26T15:09:12Z
CVE-2019-3824 ldb: version 1.5.4

* Fix standalone build of ldb.
* C99 build fixes.
* CVE-2019-3824 out of bounds read in wildcard compare (bug 13773)

Signed-off-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(v4-10-test): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(v4-10-test): Tue Feb 26 16:09:12 CET 2019 on sn-devel-144

- - - - -
a2f74069 by Jiří Šašek at 2019-03-05T19:29:10Z
notifyd: Fix SIGBUS on sparc

Problem is the structure "struct notify_instance" can lay in message buffer on
address not dividable by 8 but this structure begins by uint_64 (secs in
time-stamp). Structure should be re-packed to standalone buffer before the use.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13704
Signed-off-by: jiri.sasek at oracle.com
Reviewed-by: Volker Lendecke <vl at samba.org>

- - - - -
da238723 by Gary Lockyer at 2019-03-05T19:29:11Z
man pages: document prefork process model

Document the process model options -M

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13765

Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>

Autobuild-User(master): Tim Beale <timbeale at samba.org>
Autobuild-Date(master): Thu Jan 31 04:25:14 CET 2019 on sn-devel-144

(cherry picked from commit 4ecdcc4b462d03bec3ec36ba1b90f45d9977be8b)

- - - - -
5b7e33f0 by Volker Lendecke at 2019-03-05T19:29:11Z
messages_dgm: Use saved errno value

In this case this is just a cleanup, the value has just been set by
messaging_dgm_sendmsg. But as that already saves errno into a local
variable, use that.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13786

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit c27afc098398274abaed6dc9bef2019091c1b635)

- - - - -
d3a84988 by Volker Lendecke at 2019-03-05T19:29:11Z
torture3: Extend read3 for the "messaging target re-inits" failure

Do ping_pong a hundred times, re-initializing the msg_ctx every time.

https://bugzilla.samba.org/show_bug.cgi?id=13786

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 8d8f62c4b9dea381ce9f5833bc794553ae358173)

- - - - -
caf15a44 by Volker Lendecke at 2019-03-05T19:29:11Z
messages_dgm: Properly handle receiver re-initialization

This only properly covers the small-message nonblocking case. Covering
the large-message and the blocking case is a much larger effort assuming
we want to re-send the failed message if parts of the message has gone
through properly. Don't do that for now.

This was found by sanba_dnsupdate constantly recreating its irpc handle to
winbindd in the RODC case.

The messaging_dgm code cached connected datagram sockets based on the
destination pid for 1 second. Which means the IRPC responses from
winbindd are never delivered to samba_dnsupdate,
which will then hit a timeout.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13786

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 2543bba0364d8054e9ad316f5611621841bc061d)

- - - - -
b69bb1f7 by Martin Schwenke at 2019-03-05T19:29:11Z
ctdb-config: Change example recovery lock setting to one that fails

ctdbd will start without a recovery lock configured.  It will log a
message saying that this is not optimal.  However, a careless user may
overlook both this message and the importance of setting a recovery
lock.  If the existing example configuration is uncommented then the
directory containing it will be created (by 01.reclock.script) and the
failure (i.e. multiple nodes able to take the lock) will be confusing.

Instead, change the example setting to one that will result in banned
nodes, encouraging users to consciously configure (or deconfigure) the
recovery lock.  Tweak the corresponding comment.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13790

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 82e7f38214896c2c200132bc6dde3348cfac16cc)

- - - - -
8d28f78b by Swen Schillig at 2019-03-05T19:29:11Z
ctdb: buffer write beyond limits

In order to calculate the number of bytes correctly which
are to be read into the buffer, the buffer.offset must be taken
into account.

This patch fixes a regression introduced by 382705f495dd.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13791

Signed-off-by: Swen Schillig <swen at linux.ibm.com>
Reviewed-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>
(cherry picked from commit fa8e69ac9538980c441b7fbefe0979027ecc8eac)

- - - - -
a4a8351c by Christof Schmitt at 2019-03-05T19:29:11Z
ctdb-tests: Add test for ctdb_io.c

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13791

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Martin Schwenke <martin at meltin.net>

Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Fri Feb 22 03:51:37 CET 2019 on sn-devel-144

(cherry picked from commit 92a90524373a0348c1912d5019254dd18c07e207)

- - - - -
6ee34859 by Martin Schwenke at 2019-03-05T19:29:11Z
ctdb-recoverd: Free cluster mutex handler on failure to take lock

If nested events occur while the file descriptor handler is still
active then chaos can ensue.  For example, if a node is banned and the
lock is explicitly cancelled (e.g. due to election loss) then
double-talloc-free()s abound.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 621658cbed5d91d7096fc208bac2ff93a1880e7d)

- - - - -
04baa822 by Martin Schwenke at 2019-03-05T19:29:12Z
ctdb-recoverd: Clean up logging on failure to take recovery lock

Add an explicit case for a timeout and clean up the other messages.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 7e4aae6943291c3144c8a3ff97537e8d4c7dc7c9)

- - - - -
5a060f07 by Martin Schwenke at 2019-03-05T19:29:12Z
ctdb-recoverd: Make recoverd context available in recovery lock handle

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit c0fb62ed3954fc6e8667480aba92003fc270f257)

- - - - -
bf774b81 by Martin Schwenke at 2019-03-05T19:29:12Z
ctdb-recoverd: Ban node on unknown error when taking recovery lock

We really shouldn't see unknown errors.  They probably represent a
misconfigured recovery lock or similar.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 45a77d65b2e39b4af94da4ab99575f4ee08a7ebd)

- - - - -
d7fe8140 by Martin Schwenke at 2019-03-05T19:29:12Z
ctdb-recoverd: Time out attempt to take recovery lock after 120s

Currently this will wait forever.  It really needs a timeout in case
the cluster filesystem (or other lock mechanism) is completely wedged.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 13a1a4808935290dceb219daccd7aac3fda4e184)

- - - - -
6ac45076 by Martin Schwenke at 2019-03-05T19:29:12Z
ctdb-tests: Force test failure if local daemon setup fails

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit ce09d9c3e4c72ebec7a21686ae913398a5c9020f)

- - - - -
75fcaea7 by Martin Schwenke at 2019-03-05T19:29:12Z
ctdb-tests: Add -R option for local daemons to use recovery lock command

Under the covers, a command is always used.  However, there is no way
of testing of the code path where a command is explicitly configured.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit e74f5243fcb7594939769c16f3c79ab167dd1227)

- - - - -
c39430b0 by Martin Schwenke at 2019-03-05T19:29:12Z
ctdb-tests: Add a test for configuring the recovery lock as a command

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit ebc082122fb34ffb8cbcafde9ad39bcc241d33ed)

- - - - -
e0720c16 by Martin Schwenke at 2019-03-05T19:29:13Z
ctdb-cluster-mutex: Separate out command and file handling

This code is difficult to read and there really is no common code
between the 2 cases.  For example, there is no need to split a
filename into words.  Separating each of the 2 cases into its own
function makes the logic much easier to understand.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13800

Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Mon Feb 25 03:40:16 CET 2019 on sn-devel-144

(cherry picked from commit c93430fe8fe530a55b9a04cf6cc660c3d420e333)

- - - - -
07820a31 by Ralph Boehme at 2019-03-05T19:29:13Z
winbindd: make a copy of xid's in wb_xids2sids_send()

This is in preparation of setting the result of the mapping in the top-
level callback wb_xids2sids_done(), not in the per-idmap-domain callback
wb_xids2sids_dom_done().

When caching the mapping we need the id-type from the backend, so we
need a way to pass up that information from wb_xids2sids_dom_done() up
to wb_xids2sids_done()

The xids array copy gets passed from wb_xids2sids_send() to
wb_xids2sids_dom_send(), so wb_xids2sids_dom_done() can then directly
update the top-level copy.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit f5a8bc2f945be45cdade5f70d4f975bae8337f67)

- - - - -
f8772315 by Ralph Boehme at 2019-03-05T19:29:13Z
winbindd: make xids a const argument to wb_xids2sids_send()

The previous commit made an internal copy of xids, this commit makes it
more obvious that we must not mess with the xids argument but treat it as
an in-parameter and don't write to it.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit 5d277ea7ea258676b9ea5081a451a5874af115f6)

- - - - -
406a1cdf by Ralph Boehme at 2019-03-05T19:29:13Z
winbindd: convert id to a pointer in wb_xids2sids_dom_done()

No change in behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit f8bf4fc608639695651f75c52b31f95e796a5a26)

- - - - -
e9ffadb2 by Ralph Boehme at 2019-03-05T19:29:13Z
winbindd: update xid in wb_xids2sids_state->xids with what we got

In preparation of priming the idmap cache in the top-level
wb_xids2sids_done(), not in the per-idmap-domain callback
wb_xids2sids_dom_done().

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit 7f23ef7b2cf7bd6e8dc087aa15137292b421a689)

- - - - -
35772933 by Ralph Boehme at 2019-03-05T19:29:13Z
winbindd: switch send-next/done order

In preparation of adding more logic to the done step. No change in
behaviour.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit 8e9c2a1f6ceb06d695a6572701b96a3e3821ac42)

- - - - -
3b9ef909 by Ralph Boehme at 2019-03-05T19:29:13Z
winbindd: track whether a result from xid2sid was coming from the cache

This is needed in preparation of moving the step to update the idmap
cache from the per-idmap-domain callback wb_xids2sids_dom_done() to the
top-level callback wb_xids2sids_done().

Currently the sequence of action is:

* check cache, if not found:
  * ask backends
  * cache result from backend
* return results

Iow, if we got something from the cache, we don't write the cache.

The next commit defers updating the cache to the top-level callback, so
the sequence becomes

* check cache, if not found:
  * ask backends
* cache results
* return results

This has two problems:

* it needlessly writes to the cache what we just got from it

* it possibly overwrites the ID_TYPE_BOTH for a SID-to-xid mapping in
  the following case:

  - existing ID_TYPE_BOTH mapping in the cache, eg:

    IDMAP/SID2XID/S-1-5-21-2180672342-2513613279-2566592647-512 -> Value: 3000000:B

  - someone calls wb_xids2sids_send() with xid.id=3000000,xid.type=ID_TYPE_GID

  - cache lookup with idmap_cache_find_gid2sid() succeeds

  - when caching results we'd call idmap_cache_set_sid2unixid() with the
    callers xid.type=ID_TYPE_GID, so idmap_cache_set_sid2unixid() will
    overwrite the SID-to-xid mapping with ID_TYPE_GID

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit 62f54229fced20102e11ad1da02faef45c2a7c2e)

- - - - -
ce4c9c40 by Ralph Boehme at 2019-03-05T19:29:14Z
winbindd: set idmap cache entries as the last step in async wb_xids2sids

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13802

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Sat Feb 23 09:23:22 CET 2019 on sn-devel-144

(cherry picked from commit 9b9565c3e69b92c298c7168e516387bb249c9e36)

- - - - -
f765b515 by Jeremy Allison at 2019-03-05T19:29:14Z
s3: smbtorture3: Add POSIX-MKDIR test for posix_mkdir case sensitive bug.

Test does:

mkdir POSIX_foo
mkdir POSIX_Foo
mkdir POSIX_foo/Foo
mkdir POSIX_foo/foo
mkdir POSIX_Foo/Foo
mkdir POSIX_Foo/foo

Which should pass a SMB1 POSIX extensions server
as posix mkdir should always be case sensitive
no matter what the share is set to.

Mark as knownfail for now.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit f0decb31fff24ceb57d23bebbfb87aa0e5103947)

- - - - -
230c557e by Jeremy Allison at 2019-03-05T19:29:14Z
smbd: SMB1-POSIX: Add missing info-level SMB_POSIX_PATH_OPEN for UCF_UNIX_NAME_LOOKUP flag.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit 980db8debde68f9edce13fc01bc45a904379cffd)

- - - - -
179db55a by Jeremy Allison at 2019-03-05T19:29:14Z
smbd: unix_convert: Ensure we don't call get_real_filename on POSIX paths.

For posix_pathnames don't blunder into the name_has_wildcard OR
get_real_filename() codepaths as they may be doing case insensitive lookups.
So when creating a new POSIX directory 'Foo' they might
match on name 'foo'.

Remove POSIX-MKDIR from knownfail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Sun Feb 24 14:04:14 CET 2019 on sn-devel-144

(cherry picked from commit 12da33e2bbce657f25abe9bc4cb936ca4615b30f)

- - - - -
b58c6f1f by Jeremy Allison at 2019-03-05T19:29:14Z
s3: torture: Add additional POSIX mkdir tests.

Ensure that if POSIX_foo exists as a file
we return the correct error code NT_STATUS_OBJECT_PATH_NOT_FOUND
if we try and traverse it as a directory.

Also ensure creation/deletion of POSIX_foo/foo fails
for directories and files with NT_STATUS_OBJECT_PATH_NOT_FOUND
if the directory POSIX_foo/ doesn't exist.

knownfail is back :-).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>
(cherry picked from commit 632d0db8c42d50f5eecd002d9573f739cd945960)

- - - - -
0c034112 by Jeremy Allison at 2019-03-05T19:29:14Z
s3: smbd: filenames - ensure we replace the missing '/' if we error in an intermediate POSIX path.

Previous regression test ensures we still return the correct
error code for POSIX pathname operations.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13803

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Volker Lendecke <vl at samba.org>

Autobuild-User(master): Volker Lendecke <vl at samba.org>
Autobuild-Date(master): Mon Feb 25 09:33:27 CET 2019 on sn-devel-144

(cherry picked from commit 3f1a13a94a753c5cb3b9f2cf795df5adb0f74205)

- - - - -
0a784f28 by David Disseldorp at 2019-03-05T19:29:14Z
vfs_ceph: add missing fallocate hook

SMB_VFS_FALLOCATE() calls atop a vfs_ceph share currently fall through
to vfs_default, which results in a local filesystem I/O attempt using a
libcephfs file-descriptor. Add the missing fallocate hook to vfs_ceph.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13807

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>
(cherry picked from commit 5a7e7280813559fb70a9fc8e4238cb6015ee3b53)

- - - - -
3332a895 by David Disseldorp at 2019-03-05T19:29:14Z
vfs_ceph: fix strict_allocate_ftruncate()

The vfs_ceph "strict allocate = yes" ftruncate wrapper may attempt
*local* filesystem ftruncate(). Fix this.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13807

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>
(cherry picked from commit 70329c36ed91dd0e50ff66f9b0a85c62ac8a621e)

- - - - -
19b2885d by David Disseldorp at 2019-03-05T19:29:15Z
vfs_ceph: remove ceph_fallocate/ceph_ftruncate fallback

Both libcephfs functions are supported and capable of extending files,
so fallback can be dropped.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13807

Signed-off-by: David Disseldorp <ddiss at samba.org>
Reviewed-by: Guenther Deschner <gd at samba.org>
(cherry picked from commit 155f1289ba7a4802fbb99fbc9ea90d8bc6cff0c9)

- - - - -
3fe0c303 by Denis Cardon at 2019-03-05T19:29:15Z
fix samba-tool gpo backup syntax in WHATSNEW.txt

Signed-off-by: Denis Cardon <dcardon at tranquil.it>

- - - - -
a65f7b71 by Garming Sam at 2019-03-05T19:29:15Z
WHATSNEW: Add some detail on the changes to paged results

Signed-off-by: Garming Sam <garming at catalyst.net.nz>

- - - - -
74aa2abf by Karolin Seeger at 2019-03-05T19:29:15Z
WHATSNEW: Add release notes for Samba 4.10.0rc4.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
e399a020 by Karolin Seeger at 2019-03-05T19:29:15Z
VERSION: Disable GIT_SNAPSHOT for the 4.10.0rc4 release.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
9169e972 by Karolin Seeger at 2019-03-06T00:34:52Z
VERSION: Bump version up to 4.10.0rc5...

and re-enable GIT_SNAPSHOT.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

Autobuild-User(v4-10-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-10-test): Wed Mar  6 00:34:53 UTC 2019 on sn-devel-144

- - - - -
0e80b245 by Günther Deschner at 2019-03-06T14:00:05Z
WHATSNEW: mention new vfs_glusterfs_fuse module

Guenther

Signed-off-by: Guenther Deschner <gd at samba.org>

Autobuild-User(v4-10-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-10-test): Wed Mar  6 14:00:05 UTC 2019 on sn-devel-144

- - - - -
2f4d8214 by Andrew Bartlett at 2019-03-11T07:55:37Z
s4-server: Open and close a transaction on sam.ldb at startup

This fixes upgrading from 4.7 and earlier releases, and makes the DB
reindexing more transparent. It should also make it easier to handle
future normalisation rule changes, e.g. if we change the pack-format
of integer indexes in a future release.

Without this change, the  should have still handled reindexing the
database. We don't know why exactly this wasn't happening correctly,
but opening a transaction early in the samba process startup should
now guarantee that the DB is correctly reindexed by the time the main
samba code runs.

An alternative fix would have been to open a transaction in the the
DSDB module stack every time we connect to the database. However, this
would add an extra write lock every time we open the DB, whereas
starting samba happens much more infrequently.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13760

Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary at catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Mar  7 04:58:42 UTC 2019 on sn-devel-144

(cherry picked from commit 8b18da27cf261b0283fe66d2b827cab542488ac7)

- - - - -
ea33a7b0 by Björn Jacke at 2019-03-11T07:55:37Z
sambaundoguididx: use the right escaped oder unescaped sam ldb files

the correct filename is taken from the partition database before, we should not
unescape that because this can result in a new unescaped ldb file being created
and the script not to work at all.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13759

Signed-off-by: Bjoern Jacke <bjacke at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
(cherry picked from commit cd1ac3668cd164bd5f7cadf7b59df9541aaef83e)

- - - - -
9f85efa7 by Tim Beale at 2019-03-11T07:55:37Z
autobuild: Split backup/restore testenvs out into separate job

The samba-ad-dc-2 job was reaching its limits with the number of
testenvs and what the resource-limited CI machines can handle.
Samba processes were getting swapped out of memory, causing CI runs
to fail.

This patch splits the backup/restore testenv targets into a separate
autobuild job: samba-ad-dc-backup.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13780

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze at samba.org>

Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Tue Feb  5 12:23:31 CET 2019 on sn-devel-144

(cherry picked from commit 95b2c9d7751ae1e5a00e1fb096f045dd73c03d72)

- - - - -
49bac77e by Andrew Bartlett at 2019-03-11T07:55:37Z
autobuild: Add -py2 tests for new split backup/restore testenvs

This ensures Python2 coverage for this code while it remains supported.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13780

Signed-off-by: Andrew Bartlett <abartlet at samba.org>

- - - - -
3b52cba5 by Ralph Boehme at 2019-03-11T07:55:37Z
s4:libcli: remember return code from maximum access

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13812

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>
(cherry picked from commit 9f4ee05295827c9a607e1f63694a17906f777176)

- - - - -
ac08949d by Ralph Boehme at 2019-03-11T07:55:37Z
s4:torture: add a Maximum Access check with an Owner Rights ACE

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13812

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>
(cherry picked from commit 3ca38d2cd1189a5040e13ddab016063280be2b4d)

- - - - -
7e95499d by Ralph Boehme at 2019-03-11T07:55:37Z
libcli/security: add "Owner Rights" calculation to access_check_max_allowed()

This was missing in 44590c1b70c0a24f853c02d5fcdb3c609401e2ca.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=13812

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: David Disseldorp <ddiss at samba.org>

Autobuild-User(master): David Disseldorp <ddiss at samba.org>
Autobuild-Date(master): Thu Feb 28 19:18:16 UTC 2019 on sn-devel-144

(cherry picked from commit 5cf0764bc4b65dbc59d8626760dbe946a2234833)

- - - - -
63f0db77 by Jeremy Allison at 2019-03-11T07:55:38Z
s4:torture: Fix the test_owner_rights() test to show permissions are additive.

Tested against Windows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13812

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>
(cherry picked from commit 2e181e34c48c879235c5dc64bd7ab2b59781810c)

- - - - -
3026c1a3 by Jeremy Allison at 2019-03-11T07:55:38Z
s4:torture: Add test_owner_rights_deny().

Shows that owner and SID_OWNER_RIGHTS ACE
entries interact in max permissions requests.

Tested against Windows.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13812

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>
(cherry picked from commit fadc4c1bc5fcc3b2d9daea44ef8daf8a8ae0fbe2)

- - - - -
1dc2e296 by Ralph Boehme at 2019-03-11T07:55:38Z
libcli/security: correct access check and maximum access calculation for Owner Rights ACEs

We basically must process the Owner Rights ACEs as any other ACE wrt to the
order of adding granted permissions and checking denied permissions. According
to MS-DTYP 2.5.3.2 Owner Rights ACEs must be evaluated in the main loop over
the ACEs in an ACL and the corresponding access_mask must be directly applied
to bits_remaining. We currently defer this to after the loop over the ACEs in
ACL, this is wrong.

We just have to do some initial magic to determine if an ACL contains and
Owner Rights ACEs, and in case it doesn't we grant SEC_STD_WRITE_DAC |
SEC_STD_READ_CONTROL at the *beginning*. MS-DTYP:

-- the owner of an object is always granted READ_CONTROL and WRITE_DAC.
CALL SidInToken(Token, SecurityDescriptor.Owner, PrincipalSelfSubst)
IF SidInToken returns True THEN
   IF DACL does not contain ACEs from object owner THEN
       Remove READ_CONTROL and WRITE_DAC from RemainingAccess
       Set GrantedAccess to GrantedAccess or READ_CONTROL or WRITE_OWNER
   END IF
END IF

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13812

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 9722f75757c0e38c7f42c7cc310d56aa6eaf6392)

- - - - -
c9b6b7ed by Jeremy Allison at 2019-03-11T07:55:38Z
s4:torture: Add test_owner_rights_deny1().

Creates a 3-element ALLOW + ALLOW + DENY ACE showing that when
calculating maximum access already seen allow bits are not removed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13812

Signed-off-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Ralph Boehme <slow at samba.org>
(cherry picked from commit 0ebd8c99aed28a0ba43a22c429837f66f7e94409)

- - - - -
cc7629a2 by Ralph Boehme at 2019-03-11T07:55:38Z
s4:torture: Add test_deny1().

Creates a 2-element ALLOW + DENY ACE showing that when calculating
effective permissions and maximum access already seen allow bits are not
removed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13812

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit b205d695d769e910a91bec87451dec189ec33740)

- - - - -
5d0e2bf8 by Ralph Boehme at 2019-03-11T07:55:38Z
libcli/security: fix handling of deny type ACEs in access_check_max_allowed()

Deny ACEs must always be evaluated against explicitly granted rights
from previous ACEs.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13812

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 8d355dd9769e8990ce998b4c9f28977669b43616)

- - - - -
7bd135d2 by Joe Guo at 2019-03-11T07:55:38Z
s4/scripting/bin: open unicode files with utf8 encoding and write unicode string

In files like `libcli/util/werror_err_table.txt` and `libcli/util/ntstatus_err_table.txt`,
there were unicode quote symbols at line 6:

    ...(“this documentation”)...

In `libcli/util/wscript_build`, it will run `gen_werror.py` and `gen_ntstatus.py`
to `open` above files, read content from them and write to other files.

When encoding not specified, `open` in both python 2/3 will guess encoding from locale.

When locale is not set, it defaults to POSIX or C, and then python will use
encoding `ANSI_X3.4-1968`.

So, on a system locale is not set, `make` will fail with encoding error
for both python 2 and 3:

    File "/home/ubuntu/samba/source4/scripting/bin/gen_werror.py", line 139, in main
        errors = parseErrorDescriptions(input_file, True, transformErrorName)
      File "/home/ubuntu/samba/source4/scripting/bin/gen_error_common.py", line 52, in parseErrorDescriptions
        for line in file_contents:
      File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 318: ordinal not in range(128)

In this case, we have to use `io.open` with `encoding='utf8'`.
However, then we got unicode strs and try to write them with other strs
into new file, which means the new file must also open with utf-8 and
all other strs have to be unicode, too.

Instead of prefix `u` to all strs, a more easier/elegant way is to enable
unicode literals for the python scripts, which we normally didn't do in samba.

Since both `gen_werror.py` and `gen_ntstatus.py` are bin scripts and no
other modules import them, it should be ok for this case.

Signed-off-by: Joe Guo <joeg at catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Fri Feb  8 06:34:47 CET 2019 on sn-devel-144

(cherry picked from commit 87149445af26b8577566dfe5e311b32e3650c6e6)

- - - - -
5c2a243d by Ralph Boehme at 2019-03-11T11:50:36Z
CI: don't use swap

Signed-off-by: Ralph Boehme <slow at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>

Autobuild-User(master): Ralph Böhme <slow at samba.org>
Autobuild-Date(master): Mon Mar  4 13:59:42 UTC 2019 on sn-devel-144

(adapted from from commit 7798bc14fbdae3461eb30421923d53978b3f781d
by Andrew Bartlett)

Autobuild-User(v4-10-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-10-test): Mon Mar 11 11:50:37 UTC 2019 on sn-devel-144

- - - - -
41c1870a by Volker Lendecke at 2019-03-13T08:34:14Z
lib: Make idmap_cache return negative mappings

Without this we'd query non-existent mappings over and over
again.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit d9303e8eb90d48f09f2e2e8bdf01f4a7c3c21d11)

- - - - -
1df6720d by Volker Lendecke at 2019-03-13T08:34:14Z
idmap_cache: Only touch "sid" on success in find_xid_to_sid

Why? This makes the negative mapping condition (is_null_sid) more
explicit in the code.

The callers in lookup_sid initialized "psid" anyway before, and the ones
in wb_xids2sids now do as well. This is more in line with other APIs we
have: Only touch output parameters if you have something to say.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit 4faf3e9f6da7515fc263d79f77226d105c2f8524)

- - - - -
465bd07f by Volker Lendecke at 2019-03-13T08:34:14Z
winbind: Initialize "expired" parameter to idmap_cache_xid2sid

The code in idmap_cache only touches its output parameters upon success

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit 8c28c12702c0935a852c7fed6565987623f09fee)

- - - - -
6434de2b by Volker Lendecke at 2019-03-13T08:34:14Z
winbind: Now we explicitly track if we got ids from cache

This now properly makes us use negative cache entries

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit 95d33ca79cc315f1a2e41cd60859ef01d6548c77)

- - - - -
f6f0994a by Volker Lendecke at 2019-03-13T08:34:15Z
idmap_cache: Introduce idmap_cache_find_xid2sid

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit bb8122dd8c53bb307819a79b7888cc0940a7c13b)

- - - - -
a20e68bc by Volker Lendecke at 2019-03-13T08:34:15Z
torture: Add tests for idmap cache

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit e5a903bab6eda8f7ff2a7c8149d51022d9d8aede)

- - - - -
ba6dd781 by Volker Lendecke at 2019-03-13T08:34:15Z
winbind: Use idmap_cache_find_xid2sid

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit bc9824bd42d9370279819ea0d927e236f6041324)

- - - - -
10a0d77f by Volker Lendecke at 2019-03-13T08:34:15Z
lib: Introduce winbind_xid_to_sid

This does not merge a winbind communication error into
"global_sid_NULL" (S-1-0-0), which by the way non-intuitively does not
go along with is_null_sid(). Instead, this just touches the output sid
when winbind returned success. This success might well be a negative
mapping indicated by S-0-0, which *is* is_null_sid()...

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit ef706a3e63b3e25edd27e0f99c3e2d8ff7209cb6)

- - - - -
8c0268a5 by Volker Lendecke at 2019-03-13T08:34:15Z
passdb: Introduce xid_to_sid

This explicitly avoids the legacy_[ug]id_to_sid calls, which create
long-term cache entries to S-1-22-x-y if anthing fails. We can't do
this, because this will turn temporary winbind communication failures
into long-term problems: A short hickup in winbind_uid_to_sid will
create a mapping to S-1-22-1-uid for a week. It should be up to the
lower layers to do the caching.

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit 92f27ebb14c0c18b1d0fd49544ad851aeb14781c)

- - - - -
2021080a by Volker Lendecke at 2019-03-13T08:34:15Z
passdb: Make [ug]id_to_sid use xid_to_sid

Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Christof Schmitt <cs at samba.org>
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13813
(cherry picked from commit 40de67f1fcc46b7a64a7364c91dcedb474826d51)

- - - - -
545914af by Christof Schmitt at 2019-03-13T08:34:15Z
passdb: Update ABI to 0.27.2

This change is for the backport only. The change in master increased the
ABI version to 0.28.0 and removed some functions; this should not happen
in a backport.

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

- - - - -
3c32774b by Christof Schmitt at 2019-03-13T08:34:15Z
lib/winbind_util: Move include out of ifdef

This fixes compile errors about missing prototypes with
--picky-developer and --without-winbind

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit 4b1e4c22128bdefe549a58b181e9b755854f4c3e)

- - - - -
6d901af0 by Christof Schmitt at 2019-03-13T08:34:16Z
lib/winbind_util: Add winbind_xid_to_sid for --without-winbind

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13813

Signed-off-by: Christof Schmitt <cs at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>

Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Wed Mar  6 01:53:16 UTC 2019 on sn-devel-144

(cherry picked from commit 4125ff89e44a3e98882cfc38c06e559a6e1e56a5)

- - - - -
00ea6a7d by Andreas Schneider at 2019-03-13T12:15:10Z
lib:util: Move debug message for mkdir failing to log level 1

If you connnect to a host with smbclient this gets always printed.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13823

Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>

(cherry picked from commit c71334ec0c92e791022a9b7c900aa0dd649226c2)

Autobuild-User(v4-10-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-10-test): Wed Mar 13 12:15:10 UTC 2019 on sn-devel-144

- - - - -
5da71ca1 by Karolin Seeger at 2019-03-19T09:54:30Z
WHATSNEW: Add release notes for Samba 4.10.0.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
983bcc35 by Karolin Seeger at 2019-03-19T09:55:47Z
VERSION: Bump version up to 4.10.0.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
25f2fe02 by Karolin Seeger at 2019-03-19T09:57:00Z
VERSION: Disable GIT_SNAPSHOT for the 4.10.0 release.

Signed-off-by: Karolin Seeger <kseeger at samba.org>

- - - - -
5b2a3764 by Mathieu Parent at 2019-03-22T05:43:43Z
New upstream version 4.10.0+dfsg
- - - - -
444b3183 by Mathieu Parent at 2019-03-22T05:44:36Z
Update upstream source from tag 'upstream/4.10.0+dfsg'

Update to upstream version '4.10.0+dfsg'
with Debian dir 1539f54b7577ebc9fbf0e479b7e750334d822f49
- - - - -
4e2e9f17 by Mathieu Parent at 2019-03-22T05:58:18Z
Update patches

- - - - -
1b209f95 by Mathieu Parent at 2019-03-22T06:02:06Z
Drop nsswitch-Add-try_authtok-option-to-pam_winbind.patch, merged

- - - - -
5695ad2a by Jelmer Vernooij at 2019-03-22T06:02:10Z
Always specify rpath for private libraries

Last-Update: 2012-02-24
Applied-Upstream: no

- - - - -
c3a22dce by Christian Perrier at 2019-03-22T06:02:10Z
64 bit fix for libsmbclient

Bug-Debian: http://bugs.debian.org/221618
Forwarded: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=221618#27

- - - - -
3116fb88 by Christian Perrier at 2019-03-22T06:02:10Z
Mention smbldap-tools package in examples/LDAP/README

Bug-Debian: http://bugs.debian.org/341934
Forwarded: not-needed

- - - - -
43318471 by Steve Langasek at 2019-03-22T06:02:10Z
Use the pager alternative as pager is PAGER is undefined

Bug-Debian: http://bugs.debian.org/135603
Forwarded: not-needed

- - - - -
9f9f1427 by mathiaz at ubuntu.com at 2019-03-22T06:02:10Z
Enable net usershares by default at build time

Enable net usershares by default at build time, with a limit of 100, and update
the corresponding documentation.

Bug-Debian: http://bugs.debian.org/443230
Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/128548
Forwarded: not-needed

- - - - -
194f18bc by Eloy A. Paris at 2019-03-22T06:02:10Z
Add "Debian" as vendor suffix

Forwarded: not-needed

- - - - -
a7cc5255 by Jeroen Dekkers at 2019-03-22T06:02:10Z
Add so version number to private libraries for dpkg-shlibdeps

We also want dpkg-shlibdeps to generate correct dependency information
for the private libraries in our binary packages, but dpkg-shlibdeps
only works when the library has a version number.

Origin: vendor
Forwarded: not-needed

- - - - -
8fbdb162 by Brian May at 2019-03-22T06:02:10Z
Patch in symbol table from rfc3454, for Heimdal scripts

Status: cherry-picked from heimdal package

- - - - -
ac1215b0 by Mathieu Parent at 2019-03-22T06:02:10Z
smbd.service: Run update-apparmor-samba-profile before start

Bug-Debian: https://bugs.debian.org/896080

- - - - -


30 changed files:

- + .gitattributes
- + .github/contributing.md
- + .github/pull_request_template.md
- .gitlab-ci-private.yml
- .gitlab-ci.yml
- − .travis.yml
- .ycm_extra_conf.py
- Makefile
- README.Coding
- README → README.md
- VERSION
- WHATSNEW.txt
- auth/auth_log.c
- + auth/auth_util.c
- + auth/auth_util.h
- auth/credentials/credentials_krb5.c
- auth/credentials/pycredentials.c
- auth/credentials/tests/bind.py
- auth/credentials/wscript_build
- auth/gensec/gensec.c
- auth/ntlmssp/ntlmssp_sign.c
- auth/ntlmssp/wscript_build
- auth/wscript_build
- buildtools/bin/waf
- buildtools/examples/run_on_target.py
- buildtools/wafsamba/configure_file.py
- buildtools/wafsamba/generic_cc.py
- − buildtools/wafsamba/hpuxcc.py
- − buildtools/wafsamba/irixcc.py
- − buildtools/wafsamba/nothreads.py


The diff was not included because it is too large.


View it on GitLab: https://salsa.debian.org/samba-team/samba/compare/51996460484bcd1926bc811d2395ffbb4b249369...ac1215b015cf772cd02541bc4da2c203c7037146

-- 
View it on GitLab: https://salsa.debian.org/samba-team/samba/compare/51996460484bcd1926bc811d2395ffbb4b249369...ac1215b015cf772cd02541bc4da2c203c7037146
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-samba-maint/attachments/20190322/437c57b3/attachment-0001.html>


More information about the Pkg-samba-maint mailing list