summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)AuthorFilesLines
2018-03-19qapi: generate a literal qobject for introspectionMarc-André Lureau1-33/+43
Replace the generated json string with a literal qobject. The later is easier to deal with, at run time as well as compile time: adding #if conditionals will be easier than in a json string. The output of query-qmp-schema is not changed. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180305172951.2150-5-marcandre.lureau@redhat.com> [eblake: fix python 3 failure] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19qapi2texi: minor python code simplificationMarc-André Lureau1-4/+3
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180305172951.2150-2-marcandre.lureau@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-16Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into stagingPeter Maydell2-1/+310
* Record-replay lockstep execution, log dumper and fixes (Alex, Pavel) * SCSI fix to pass maximum transfer size (Daniel Barboza) * chardev fixes and improved iothread support (Daniel Berrangé, Peter) * checkpatch tweak (Eric) * make help tweak (Marc-André) * make more PCI NICs available with -net or -nic (myself) * change default q35 NIC to e1000e (myself) * SCSI support for NDOB bit (myself) * membarrier system call support (myself) * SuperIO refactoring (Philippe) * miscellaneous cleanups and fixes (Thomas) # gpg: Signature made Mon 12 Mar 2018 16:10:52 GMT # gpg: using RSA key BFFBD25F78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" # Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1 # Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83 * remotes/bonzini/tags/for-upstream: (69 commits) tcg: fix cpu_io_recompile replay: update documentation replay: save vmstate of the asynchronous events replay: don't process async events when warping the clock scripts/replay-dump.py: replay log dumper replay: avoid recursive call of checkpoints replay: check return values of fwrite replay: push replay_mutex_lock up the call tree replay: don't destroy mutex at exit replay: make locking visible outside replay code replay/replay-internal.c: track holding of replay_lock replay/replay.c: bump REPLAY_VERSION again replay: save prior value of the host clock replay: added replay log format description replay: fix save/load vm for non-empty queue replay: fixed replay_enable_events replay: fix processing async events cpu-exec: fix exception_index handling hw/i386/pc: Factor out the superio code hw/alpha/dp264: Use the TYPE_SMC37C669_SUPERIO ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org> # Conflicts: # default-configs/i386-softmmu.mak # default-configs/x86_64-softmmu.mak
2018-03-16Merge remote-tracking branch 'remotes/awilliam/tags/vfio-update-20180313.0' ↵Peter Maydell1-0/+5
into staging VFIO updates 2018-03-13 - Display support for vGPUs (Gerd Hoffmann) - Enable new kernel support for mmaps overlapping MSI-X vector table, disable MSI-X emulation on POWER (Alexey Kardashevskiy) # gpg: Signature made Tue 13 Mar 2018 19:48:49 GMT # gpg: using RSA key 239B9B6E3BB08B22 # gpg: Good signature from "Alex Williamson <alex.williamson@redhat.com>" # gpg: aka "Alex Williamson <alex@shazbot.org>" # gpg: aka "Alex Williamson <alwillia@redhat.com>" # gpg: aka "Alex Williamson <alex.l.williamson@gmail.com>" # Primary key fingerprint: 42F6 C04E 540B D1A9 9E7B 8A90 239B 9B6E 3BB0 8B22 * remotes/awilliam/tags/vfio-update-20180313.0: ppc/spapr, vfio: Turn off MSIX emulation for VFIO devices vfio-pci: Allow mmap of MSIX BAR vfio/pci: Relax DMA map errors for MMIO regions vfio/display: adding dmabuf support vfio/display: adding region support vfio/display: core & wireup vfio/common: cleanup in vfio_region_finalize secondary-vga: properly close QemuConsole on unplug console: minimal hotplug suport ui/pixman: add qemu_drm_format_to_pixman() standard-headers: add drm/drm_fourcc.h Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-15Merge remote-tracking branch ↵Peter Maydell1-13/+24
'remotes/vivier2/tags/linux-user-for-2.12-pull-request' into staging # gpg: Signature made Tue 13 Mar 2018 17:33:03 GMT # gpg: using RSA key F30C38BD3F2FBE3C # gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" # gpg: aka "Laurent Vivier <laurent@vivier.eu>" # gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" # Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C * remotes/vivier2/tags/linux-user-for-2.12-pull-request: linux-user: init_guest_space: Add a comment about search strategy linux-user: init_guest_space: Don't try to align if we'll reject it linux-user: init_guest_space: Clean up control flow a bit linux-user: init_guest_commpage: Add a comment about size check linux-user: init_guest_space: Clarify page alignment logic linux-user: init_guest_space: Correctly handle guest_start in commpage initialization linux-user: init_guest_space: Clean up if we can't initialize the commpage linux-user: Rename validate_guest_space => init_guest_commpage linux-user: Use #if to only call validate_guest_space for 32-bit ARM target qemu-binfmt-conf.sh: add qemu-xtensa linux-user: drop unused target_msync function linux-user: fix target_mprotect/target_munmap error return values linux-user: fix assertion in shmdt linux-user: fix mmap/munmap/mprotect/mremap/shmat linux-user: Support f_flags in statfs when available. linux-user: allows to use "--systemd ALL" with qemu-binfmt-conf.sh linux-user: Remove the unused "not implemented" signal handling stubs linux-user: Drop unicore32 code Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-15Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream-sev' into ↵Peter Maydell1-1/+4
staging * Migrate MSR_SMI_COUNT (Liran) * Update kernel headers (Gerd, myself) * SEV support (Brijesh) I have not tested non-x86 compilation, but I reordered the SEV patches so that all non-x86-specific changes go first to catch any possible issues (which weren't there anyway :)). # gpg: Signature made Tue 13 Mar 2018 16:37:06 GMT # gpg: using RSA key BFFBD25F78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" # Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1 # Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83 * remotes/bonzini/tags/for-upstream-sev: (22 commits) sev/i386: add sev_get_capabilities() sev/i386: qmp: add query-sev-capabilities command sev/i386: qmp: add query-sev-launch-measure command sev/i386: hmp: add 'info sev' command cpu/i386: populate CPUID 0x8000_001F when SEV is active sev/i386: add migration blocker sev/i386: finalize the SEV guest launch flow sev/i386: add support to LAUNCH_MEASURE command target/i386: encrypt bios rom sev/i386: add command to encrypt guest memory region sev/i386: add command to create launch memory encryption context sev/i386: register the guest memory range which may contain encrypted data sev/i386: add command to initialize the memory encryption context include: add psp-sev.h header file sev/i386: qmp: add query-sev command target/i386: add Secure Encrypted Virtualization (SEV) object kvm: introduce memory encryption APIs kvm: add memory encryption context docs: add AMD Secure Encrypted Virtualization (SEV) machine: add memory-encryption option ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-13standard-headers: add drm/drm_fourcc.hGerd Hoffmann1-0/+5
So we can use the drm fourcc codes without a dependency on libdrm-devel. Signed-off-by: Gerd Hoffmann <kraxel@redhat.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2018-03-13Merge remote-tracking branch ↵Peter Maydell3-3/+7
'remotes/ehabkost/tags/python-next-pull-request' into staging Python queue, 2018-03-12 # gpg: Signature made Mon 12 Mar 2018 22:10:36 GMT # gpg: using RSA key 2807936F984DC5A6 # gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" # Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6 * remotes/ehabkost/tags/python-next-pull-request: device-crash-test: Use 'python' binary qmp.py: Encode json data before sending qemu.py: Use items() instead of iteritems() device-crash-test: New known crashes Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-13include: add psp-sev.h header fileBrijesh Singh1-1/+1
The header file provide the ioctl command and structure to communicate with /dev/sev device. Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Richard Henderson <rth@twiddle.net> Cc: Eduardo Habkost <ehabkost@redhat.com> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
2018-03-13update Linux headers to 4.16-rc5Paolo Bonzini1-0/+3
Note that VIRTIO_GPU_CAPSET_VIRGL2 was added manually so it has to be added manually after re-running scripts/update-linux-headers.sh. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-03-13Merge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' ↵Peter Maydell4-13/+60
into staging # gpg: Signature made Mon 12 Mar 2018 15:59:54 GMT # gpg: using RSA key 9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/tracing-pull-request: trace: only permit standard C types and fixed size integer types trace: remove use of QEMU specific types from trace probes trace: include filename when printing parser error messages simpletrace: fix timestamp argument type log-for-trace.h: Split out parts of log.h used by trace.h Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-12device-crash-test: Use 'python' binaryEduardo Habkost1-1/+1
Now the script works with Python 3, so we can use the 'python' binary provided by the system. Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <20180312185503.5746-4-ehabkost@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-03-12qmp.py: Encode json data before sendingEduardo Habkost1-1/+1
On Python 3, json.dumps() return a str object, which can't be sent directly through a socket and must be encoded into a bytes object. Use .encode('utf-8'), which will work on both Python 2 and Python 3. Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <20180312185503.5746-3-ehabkost@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-03-12qemu.py: Use items() instead of iteritems()Eduardo Habkost1-1/+1
items() is less efficient on Python 2.x, but makes the code work on both Python 2 and Python 3. Cc: Lukáš Doktor <ldoktor@redhat.com> Cc: Philippe Mathieu-Daudé <f4bug@amsat.org> Cc: Cleber Rosa <crosa@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <20180312185503.5746-2-ehabkost@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-03-12device-crash-test: New known crashesEduardo Habkost1-0/+4
We are not running the script on "make check" yet, and additional bugs were introduced recently in the tree. Whitelist the new crashes while we investigate, to allow us to run device-crash-test on "make check" as soon as possible to prevent new bugs. Cc: Pavel Pisa <pisa@cmp.felk.cvut.cz> Cc: John Snow <jsnow@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <20180309202827.12085-5-ehabkost@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-03-12scripts/replay-dump.py: replay log dumperAlex Bennée1-0/+308
This script is a debugging tool for looking through the contents of a replay log file. It is incomplete but should fail gracefully at events it doesn't understand. It currently understands two different log formats as the audio record/replay support was merged during since MTTCG. It was written to help debug what has caused the BQL changes to break replay support. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20180227095310.1060.14500.stgit@pasha-VirtualBox> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-03-12checkpatch: Exempt long URLsEric Blake1-1/+2
Sometimes, we want to refer to really long URLs, but checkpatch balks, and we have to manually bypass the check. URL shorteners may be nice at reducing long links, but it's hard to guarantee the shortened link will live as long as the real target, and it is also nice to see the original target without having to load the shortened URL through a browser. So exempt a line containing only a URL from the long-line syntax check. Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20180222215838.18223-1-eblake@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-03-12trace: only permit standard C types and fixed size integer typesDaniel P. Berrangé1-0/+46
Some trace backends will compile code based on the declared trace events. It should not be assumed that the backends can resolve any QEMU specific typedefs. So trace events should restrict their argument types to the standard C types and fixed size integer types. Any complex pointer types can be declared as "void *" for purposes of trace events, since nothing will be dereferencing these pointer arguments. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20180308155524.5082-3-berrange@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-03-12trace: include filename when printing parser error messagesDaniel P. Berrangé3-5/+7
Improves error messages from: ValueError: Error on line 72: need more than 1 value to unpack To ValueError: Error at /home/berrange/src/virt/qemu/trace-events:72: need more than 1 value to unpack Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-id: 20180306154650.24075-1-berrange@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-03-12simpletrace: fix timestamp argument typeStefan Hajnoczi1-1/+1
The timestamp argument to a trace event method is documented as follows: The method can also take a timestamp argument before the trace event arguments: def runstate_set(self, timestamp, new_state): ... Timestamps have the uint64_t type and are in nanoseconds. In reality methods with a timestamp argument actually receive a tuple like (123456789,) as the timestamp argument. This is due to a bug in simpletrace.py. This patch unpacks the tuple so that methods receive the correct timestamp argument type. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20180222163901.14095-1-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-03-12log-for-trace.h: Split out parts of log.h used by trace.hPeter Maydell1-7/+6
A persistent build problem we see is where a source file accidentally omits the #include of log.h. This slips through local developer testing because if you configure with the default (log) trace backend trace.h will pull in log.h for you. Compilation fails only if some other backend is selected. To make this error cause a compile failure regardless of the configured trace backend, split out the parts of log.h that trace.h requires into a new log-for-trace.h header. Since almost all manual uses of the log.h functions will use constants or functions which aren't in log-for-trace.h, this will let us catch missing #include "qemu/log.h" more consistently. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20180213140029.8308-1-peter.maydell@linaro.org Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-03-12audio: add driver registryGerd Hoffmann1-1/+1
Add registry for audio drivers, using the existing audio_driver struct. Make all drivers register themself. The old list of audio_driver struct pointers is now a list of audio driver names, specifying the priority (aka probe order) in case no driver is explicitly asked for. Signed-off-by: Gerd Hoffmann <kraxel@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-id: 20180306074053.22856-2-kraxel@redhat.com
2018-03-09qemu-binfmt-conf.sh: add qemu-xtensaMax Filippov1-2/+10
Register qemu-xtensa and qemu-xtensaeb for transparent linux userspace emulation. Cc: Riku Voipio <riku.voipio@iki.fi> Cc: Laurent Vivier <laurent@vivier.eu> Signed-off-by: Max Filippov <jcmvbkbc@gmail.com> Reviewed-by: Laurent Vivier <laurent@vivier.eu> Message-Id: <20180228221609.11265-11-jcmvbkbc@gmail.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2018-03-09linux-user: allows to use "--systemd ALL" with qemu-binfmt-conf.shLaurent Vivier1-11/+14
qemu-binfmt-conf.sh when it is used with systemd needs to know for which CPU the systemd-binfmt.service file must be created (i.e. "--systemd ppc"). But sometime, for instance for test purpose, we need to create an entry for all known architectures. This patch entroduce the "ALL" parameter for this purpose. Signed-off-by: Laurent Vivier <laurent@vivier.eu> Message-Id: <20180308104859.3315-1-laurent@vivier.eu>
2018-03-07RISC-V Build InfrastructureMichael Clark1-1/+12
This adds RISC-V into the build system enabling the following targets: - riscv32-softmmu - riscv64-softmmu - riscv32-linux-user - riscv64-linux-user This adds defaults configs for RISC-V, enables the build for the RISC-V CPU core, hardware, and Linux User Emulation. The 'qemu-binfmt-conf.sh' script is updated to add the RISC-V ELF magic. Expected checkpatch errors for consistency reasons: ERROR: line over 90 characters FILE: scripts/qemu-binfmt-conf.sh Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Sagar Karandikar <sagark@eecs.berkeley.edu> Signed-off-by: Michael Clark <mjc@sifive.com>
2018-03-06checkpatch: add check for `while` and `for`Su Hang1-2/+3
Adding check for `while` and `for` statements, which condition has more than one line. The former checkpatch.pl can check `if` statement, which condition has more than one line, whether block misses brace round, like this: ''' if (cond1 || cond2) statement; ''' But it doesn't do the same check for `for` and `while` statements. Using `(?:...)` instead of `(...)` in regex pattern catch. Because `(?:...)` is faster and avoids unwanted side-effect. Suggested-by: Stefan Hajnoczi <stefanha@redhat.com> Suggested-by: Eric Blake <eblake@redhat.com> Suggested-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Su Hang <suhang16@mails.ucas.ac.cn> Message-Id: <1520319890-19761-1-git-send-email-suhang16@mails.ucas.ac.cn> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-03-06checkpatch: add a warning for basename/dirnameJulia Suvorova1-0/+5
g_path_get_* do the same as g_strdup(basename/dirname(...)) but without modifying the argument. Signed-off-by: Julia Suvorova <jusual@mail.ru> Message-Id: <1519987399-19160-1-git-send-email-jusual@mail.ru> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-03-02qapi: Move qapi-schema.json to qapi/, rename generated filesMarkus Armbruster6-26/+11
Move qapi-schema.json to qapi/, so it's next to its modules, and all files get generated to qapi/, not just the ones generated for modules. Consistently name the generated files qapi-MODULE.EXT: qmp-commands.[ch] become qapi-commands.[ch], qapi-event.[ch] become qapi-events.[ch], and qmp-introspect.[ch] become qapi-introspect.[ch]. This gets rid of the temporary hacks in scripts/qapi/commands.py, scripts/qapi/events.py, and scripts/qapi/common.py. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-28-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: Fix trailing dot in tpm.c, undo temporary hack for OSX toolchain] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02Include less of the generated modular QAPI headersMarkus Armbruster4-17/+25
In my "build everything" tree, a change to the types in qapi-schema.json triggers a recompile of about 4800 out of 5100 objects. The previous commit split up qmp-commands.h, qmp-event.h, qmp-visit.h, qapi-types.h. Each of these headers still includes all its shards. Reduce compile time by including just the shards we actually need. To illustrate the benefits: adding a type to qapi/migration.json now recompiles some 2300 instead of 4800 objects. The next commit will improve it further. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-24-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> [eblake: rebase to master] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Generate separate .h, .c for each moduleMarkus Armbruster3-21/+66
Our qapi-schema.json is composed of modules connected by include directives, but the generated code is monolithic all the same: one qapi-types.h with all the types, one qapi-visit.h with all the visitors, and so forth. These monolithic headers get included all over the place. In my "build everything" tree, adding a QAPI type recompiles about 4800 out of 5100 objects. We wouldn't write such monolithic headers by hand. It stands to reason that we shouldn't generate them, either. Split up generated qapi-types.h to mirror the schema's modular structure: one header per module. Name the main module's header qapi-types.h, and sub-module D/B.json's header D/qapi-types-B.h. Mirror the schema's includes in the headers, so that qapi-types.h gets you everything exactly as before. If you need less, you can include one or more of the sub-module headers. To be exploited shortly. Split up qapi-types.c, qapi-visit.h, qapi-visit.c, qmp-commands.h, qmp-commands.c, qapi-event.h, qapi-event.c the same way. qmp-introspect.h, qmp-introspect.c and qapi.texi remain monolithic. The split of qmp-commands.c duplicates static helper function qmp_marshal_output_str() in qapi-commands-char.c and qapi-commands-misc.c. This happens when commands returning the same type occur in multiple modules. Not worth avoiding. Since I'm going to rename qapi-event.[ch] to qapi-events.[ch], and qmp-commands.[ch] to qapi-commands.[ch], name the shards that way already, to reduce churn. This requires temporary hacks in commands.py and events.py. Similarly, c_name() must temporarily be taught to munge '/' in common.py. They'll go away with the rename. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-23-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: declare a dummy variable in each .c file, to shut up OSX toolchain warnings about empty .o files, including hacking c_name()] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi/common: Fix guardname() for funny filenamesMarkus Armbruster1-1/+1
guardname() fails to return a valid C identifier for arguments containing anything but [A-Za-z0-9_.-']. Fix that. Don't bother protecting ticklish identifiers; header guards are all-caps, and no ticklish identifiers are. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180211093607.27351-22-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi/types qapi/visit: Generate built-in stuff into separate filesMarkus Armbruster3-83/+102
Linking code from multiple separate QAPI schemata into the same program is possible, but involves some weirdness around built-in types: * We generate code for built-in types into .c only with option --builtins. The user is responsible for generating code for exactly one QAPI schema per program with --builtins. * We generate code for built-in types into .h regardless of --builtins, but guarded by #ifndef QAPI_VISIT_BUILTIN. Because all copies of this code are exactly the same, including any combination of these headers works. Replace this contraption by something more conventional: generate code for built-in types into their very own files: qapi-builtin-types.c, qapi-builtin-visit.c, qapi-builtin-types.h, qapi-builtin-visit.h, but only with --builtins. Obey --output-dir, but ignore --prefix for them. Make qapi-types.h include qapi-builtin-types.h. With multiple schemata you now have multiple qapi-types.[ch], but only one qapi-builtin-types.[ch]. Same for qapi-visit.[ch] and qapi-builtin-visit.[ch]. Bonus: if all you need is built-in stuff, you can include a much smaller header. To be exploited shortly. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-21-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: fix octal constant for python 3] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Make code-generating visitors use QAPIGen moreMarkus Armbruster7-243/+189
The use of QAPIGen is rather shallow so far: most of the output accumulation is not converted. Take the next step: convert output accumulation in the code-generating visitor classes. Helper functions outside these classes are not converted. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-20-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: rebase to earlier guardstart cleanup] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Rename generated qmp-marshal.c to qmp-commands.cMarkus Armbruster1-1/+1
All generated .c are named like their .h, except for qmp-marshal.c and qmp-commands.h. To add to the confusion, tests-qmp-commands.c falsely matches generated test-qmp-commands.h. Get rid of this unnecessary complication. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-19-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Record 'include' directives in intermediate representationMarkus Armbruster1-4/+39
The include directive permits modular QAPI schemata, but the generated code is monolithic all the same. To permit generating modular code, the front end needs to pass more information on inclusions to the back ends. The commit before last added the necessary information to the parse tree. This commit adds it to the intermediate representation and its QAPISchemaVisitor. A later commit will use this to to generate modular code. New entity QAPISchemaInclude represents inclusions. Call new visitor method visit_include() for it, so visitors can see the sub-modules a module includes. Note that unlike other entities, QAPISchemaInclude has no name, and is therefore not added to entity_dict. New QAPISchemaEntity attribute @module names the entity's source file. Call new visitor method visit_module() when it changes during a visit, so visitors can keep track of the module being visited. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-18-armbru@redhat.com> [eblake: avoid accidental deletion of self._predefining] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Generate in source orderMarkus Armbruster1-2/+4
The generators' conversion to visitors (merge commit 9e72681d16) changed the processing order of entities from source order to alphabetical order. The next commit needs source order, so change it back. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-17-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Record 'include' directives in parse treeMarkus Armbruster1-4/+16
The parse tree is a list of expressions. Except include expressions currently get replaced by the included file's parse tree. Instead of throwing away the include expression, keep it with the file name expanded so you don't have to track the including file's directory to make sense of it. A future commit will put this include expression to use. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-16-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: fix check of expr after assignment] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()Markus Armbruster1-6/+9
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-15-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Lift error reporting from QAPISchema.__init__() to callersMarkus Armbruster2-16/+15
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-14-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi/common: Eliminate QAPISchema.exprsMarkus Armbruster1-4/+4
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-13-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Improve include file name reporting in error messagesMarkus Armbruster1-6/+6
Error messages print absolute file names of included files even if the user gave a relative one on the command line: $ PYTHONPATH=scripts python -B tests/qapi-schema/test-qapi.py tests/qapi-schema/include-cycle.json In file included from tests/qapi-schema/include-cycle.json:1: In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1: /work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json Improve this to In file included from tests/qapi-schema/include-cycle.json:1: In file included from tests/qapi-schema/include-cycle-b.json:1: tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json The error message when an include file can't be opened prints the include directive's file name, which is relative to the including file. Change this to print the file name relative to the working directory. Visible in tests/qapi-schema/include-no-file.err. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-12-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Touch generated files only when they changeMarkus Armbruster1-2/+9
A massive number of objects depends on QAPI-generated headers. In my "build everything" tree, it's roughly 4800 out of 5100. This is particularly annoying when only some of the generated files change, say for a doc fix. Improve qapi-gen.py to touch its output files only if they actually change. Rebuild time for a QAPI doc fix drops from many minutes to a few seconds. Rebuilds get faster for certain code changes, too. For instance, adding a simple QMP event now recompiles less than 200 instead of 4800 objects. But adding a QAPI type is as bad as ever; we've clearly got more work to do. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180211093607.27351-11-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: fix octal constant for python3] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi-gen: Convert from getopt to argparseMarkus Armbruster2-64/+33
argparse is nicer to use than getopt, and gives us --help almost for free. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-10-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> [eblake: Fix --output-dir editing accident] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi-gen: New common driver for code and doc generatorsMarkus Armbruster9-141/+73
Whenever qapi-schema.json changes, we run six programs eleven times to update eleven files. Similar for qga/qapi-schema.json. This is silly. Replace the six programs by a single program that spits out all eleven files. The programs become modules in new Python package qapi, along with the helper library. This requires moving them to scripts/qapi/. While moving them, consistently drop executable mode bits. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-9-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: move change to one-line 'blurb' earlier in series, mention mode bit change as intentional, update qapi-code-gen.txt to match actual generated events.c file] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Turn generators into modulesMarkus Armbruster5-115/+143
The next commit will introduce a common driver program for all generators. The generators need to be modules for that. qapi2texi.py already is. Make the other generators follow suit. The changes are actually trivial. Obvious in the diffs once you view them with whitespace changes ignored. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-8-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: minor tweak to keep 'blurb' one line] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Reduce use of global variables in generators someMarkus Armbruster5-31/+34
In preparation of the next commit, which will turn the generators into modules. These global variables will become local to main() then. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-7-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDocMarkus Armbruster6-94/+113
These classes encapsulate accumulating and writing output. Convert C code generation to QAPIGenC and QAPIGenH. The conversion is rather shallow: most of the output accumulation is not converted. Left for later. The indentation machinery uses a single global variable indent_level, even though we generally interleave creation of a .c and its .h. It should become instance variable of QAPIGenC. Also left for later. Documentation generation isn't converted, and QAPIGenDoc isn't used. This will change shortly. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-6-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: fix nits spotted by Michael] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Rename variable holding the QAPISchemaGenFOOVisitorMarkus Armbruster5-20/+20
Rename the variable holding the QAPISchemaGenFOOVisitor from gen to vis, to avoid confusion in the next commit. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-5-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Generate up-to-date copyright noticeMarkus Armbruster6-104/+80
Each generator carries a copyright notice for the generator itself, and another one for the files it generates. Only the former have been updated along the way, the latter have not, and are all out of date. Fix by copying the generator's copyright notice to the generated files instead. Note that the fix doesn't copy the "Authors:" part; the generated files' outdated Authors list goes away without replacement. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-4-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: Flatten each 'blurb' to one line] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Streamline boilerplate comment generationMarkus Armbruster7-125/+34
Every generator has separate boilerplate for .h and .c, and their differences are boring. All of them repeat the license note. Reduce the repetition as follows. Move common text like the license note to common open_output(), next to the existing common text there. For each generator, replace the two separate descriptions by a single one. While there, emit an "automatically generated" note into generated documentation, too. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-3-armbru@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-30 18:53:48 +0000