About direct_irq_injection and the wip/direct-irq-injection branch

[Sunt-git]

Hello author, I would like to know the relevant features of the two branches of direct_irq_injection and the wip/direct-irq-injection. Can they skip the virtual cpu interface and inject physical interrupts directly into the Guest OS?

[josecm]

Direct injection is a feature that indeed " skips the virtual CPU interface and injects physical interrupts directly into the Guest OS". The real GIC CPU interface is assigned to the guest and IRQs are routed to EL1 (as opposed to EL2), but accesses to the virtual distributor/redistributor are still trapped and emulated. In this way, interrupts are delivered directly to guests with actual native latencies. Then for the hypervisor still be able to receive interrupts (mainly IPIs) we either redirect FIQs to EL2 and add a TF-A service to configure GIC group 0 interrupts via SMC calls, or make use of the Arm SDEI infrastructure. That is the difference between the two branches.

PS: I moved this do the discussions. Next time, please open this kind of thread there.

[Sunt-git]

I see that direct irq injection in the code is only implemented in GICv2. I don't know if it can be implemented in GICv3？

[Sunt-git]

Thanks for your reply, I probably understand. I want to support direct interrupt injection on GICv3 (and maybe I want to add GICv4 support in bao later), can I refer to the code of direct interrupt injection in GICv3?

[josecm]

We'd welcome your contribution! But I didn't really understand what you meant by "refer to the code of direct interrupt injection in GICv3".

[Sunt-git]

I thought that you have already implemented the code for direct interrupts injection of GICv3, so currently only the code for direct interrupts injection of GICv2. To implement such a function on v3, is it possible to refer to the implementation of v2?

[josecm]

Yes, that's correct. And yes, it should be pretty similar. The main difference should relate to the fact that in GICv3 the CPU interface is not MMIO but part of the sysregs. I'll update this thread if any other issues come to mind.

[Sunt-git]

Alright, thank you very much!

[JorgeMVP]

Interesting discussion.

• In case of GICv3 and in a scenario with a platform of 4 pCPUS where we have a guest running in the first pair and other in the second pair.

• How can we prevent a guest from sending unexpected SGIs to the other one? Since SGI is sent using ICC_SG1R_EL1 from the CPU interface?

What about the GICR should be also given to the guest or it is too powerfull and has to be part of the Hypervisor?

[josecm]

• How can we prevent a guest from sending unexpected SGIs to the other one? Since SGI is sent using ICC_SG1R_EL1 from the CPU interface?

This is a really good point. It seems there is no straightforward way of guaranteeing isolation from SGIs on GICv3 under direct injection.

[JorgeMVP]

For the use-case where we have a primary VM with multicore and single-core service VMs it seems perfectly fine.

Single-core OSes most likely do not use SGIs ;) But then we need to disable the SGI1R from the CPU interface somehow. Or we can also consider that those services VMs are trusted VMs.

https://developer.arm.com/documentation/ddi0595/2021-06/AArch64-Registers/ICH-HCR-EL2--Interrupt-Controller-Hyp-Control-Register

if PSTATE.EL == EL0 then
    UNDEFINED;
elsif PSTATE.EL == EL1 then
    if Halted() && HaveEL(EL3) && EDSCR.SDD == '1' && boolean IMPLEMENTATION_DEFINED "EL3 trap priority when SDD == '1'" && SCR_EL3.<IRQ,FIQ> == '11' then
        UNDEFINED;
    elsif ICC_SRE_EL1.SRE == '0' then
        AArch64.SystemAccessTrap(EL1, 0x18);
    elsif EL2Enabled() && ICH_HCR_EL2.TC == '1' then
        AArch64.SystemAccessTrap(EL2, 0x18);
    elsif EL2Enabled() && HCR_EL2.FMO == '1' then
        AArch64.SystemAccessTrap(EL2, 0x18);
    elsif EL2Enabled() && HCR_EL2.IMO == '1' then
        AArch64.SystemAccessTrap(EL2, 0x18);
    elsif HaveEL(EL3) && SCR_EL3.<IRQ,FIQ> == '11' then
        if Halted() && EDSCR.SDD == '1' then
            UNDEFINED;
        else
            AArch64.SystemAccessTrap(EL3, 0x18);
    else
        ICC_SGI1R_EL1 = X[t];

It seems it is possible to trap accesses to ICC_SG1R_EL1 but not alone, and if you trap the others you lose the purpose of giving direct access of the CPU interface to the vcpu. Look at ICH_HCR_EL2.

[josecm]

It seems it is possible to trap accesses to ICC_SG1R_EL1 but not alone, and if you trap the others you lose the purpose of giving direct access of the CPU interface to the vcpu. Look at ICH_HCR_EL2.

I don't think we lose the true purpose (i.e. direct injection and handling without hypervisor intervention). The other affected registers, (ICC_CTLR_EL1, ICC_DIR_EL1, ICC_PMR_EL1, ICC_RPR_EL1) are not likely to be accessed during interrupt handling for most OSes. They could be just passed-through when an access to them generates a trap.

[josecm]

I've implemented this for cortex-R (which does not have EL3 mode) and it seems to work fine. We do have to emulate ICC_CTLR_EL1, ICC_DIR_EL1, ICC_PMR_EL1, ICC_RPR_EL1. I've implemented this by dividing the priority space in two and allowing the guest only to set interrupt priorities and PMR in the lower half and translating RPR to the correct value. Also not supporting decoupled prior drop and deactivation, since the hypervisor does not need it if not injecting interrupts.